Most production LLM control sits between two extremes: prompt engineering (brittle, context-dependent) and fine-tuning (expensive, slow iteration). Mentat, a YC F24 launch, introduces a third path: runtime intervention that modifies token probabilities mid-generation without retraining weights. For financial agents that need deterministic behavior and auditable reasoning, this matters.
The core claim is simple. You send a request to their API with steering rules, and the model adjusts its reasoning path in real time. No gradient descent. No dataset curation. No waiting for training runs.
How Runtime Steering Works
Traditional inference generates tokens by sampling from a probability distribution over the vocabulary. Runtime intervention modifies that distribution before sampling, based on rules you define.
The mechanics:
