Decision models (2): what speed changes

In the first part of this article, we introduced Jev, TypeSafe AI’s model that cannot write a sentence, and SemIf, the open replica that reads the same capability out of a four-billion-parameter model. A decision model receives a situation and a list of options, and it returns a probability for each option, in a single pass, in about a hundred milliseconds. The categories are no longer fixed inside the model; they are written into the program.

So far, you could think of Jev or SemIf as more flexible classifiers. That would miss the point. When the response time drops from several seconds to a few hundred milliseconds, the uses change in nature. This second part looks at what this speed makes possible, and what it does not excuse you from doing.

What speed changes

TypeSafe demonstrates this with the game Doom. The model continuously receives a structured description of the game state, enemy positions, distances, angles, and chooses an action. The demo runs at about ten decisions per second for roughly seven dollars of compute per hour. It should be noted, as several commentators pointed out, that the model reads coordinates rather than images: it has access to information a human player does not. The demo still illustrates what a fast loop makes possible.

On financial markets, an engine of this kind can continuously reassess an exposure or a set of positions; at the scale of a second, a minute, or an hour, where the strategies we automate operate, a hundred milliseconds no longer matters. In robotics or driver assistance, more caution is needed: low-level control loops will not be entrusted to a model that decides approximately where to steer. But a tactical layer, the one that chooses between slowing down, staying in lane, or aborting a maneuver, can work with this, provided a deterministic controller then decides exactly how to carry out the choice.

A fast decision mainly makes it possible to compare several

This is where the change of scale becomes interesting. When each evaluation costs ten seconds, you ask the model one question and accept its answer. When it costs a hundred milliseconds, you can afford to test several hypotheses before acting.

Imagine three possible actions, scored by the model. If you have a simulator able to estimate what each one produces, you can in turn score the situations that result from them:

We are no longer just asking what the best decision is right now. We are asking what happens if we take this decision, then that one. This is the same principle found, under different names, in planning, predictive control, and the tree search used by game-playing programs.

One limitation remains. A decision model is good at comparing the options it is given, but it does not invent any. If we offer A, B, C, and D, it will say that B is preferable; it will not spontaneously suggest a solution E that is better than all four. Hence a pairing that emerges in several recent pieces of work: a slow generative model proposes candidates, and a fast decision model scores them and keeps only the best, which are then varied further.

Slow generation is no longer called at every step; it is used to explore. Research on answer verifiers makes a similar point: a judge that writes out its reasoning becomes a bottleneck as soon as it has to score many candidates, and a fast scorer takes its place wherever possible. The generator, for that matter, can be something other than a language model: an evolutionary algorithm, a physical simulation, a business rules engine, or simple programmed variations.

A loop that tries several solutions and keeps the best one is doing optimization, not yet reinforcement learning. To call it learning, you need to observe a reward after the action and use it to improve future decisions. The speed of a decision model does not provide that step; it makes it feasible, because a policy that can be queried ten times a second can also be evaluated ten times a second.

The small model comes back into play

There is another, more direct consequence for automating a business process. At launch, the company has no usable dataset, and a general-purpose decision model can serve from day one. Every run can then be saved: the initial state, the proposed decision, its confidence level, any correction made by a person, and the observed outcome.

After a few months, you have something more valuable than a general-purpose model: data specific to your own process. You can then train a tiny specialized model that handles the repetitive cases it knows very well, and leave the general-purpose model to deal with rare cases, new categories, and anything that does not resemble what it has already seen. The literature on model cascades, opened up by FrugalGPT in 2023, measures what this arrangement delivers: the accuracy of the best model for a fraction of its cost, because most cases never need it at all.

Two precautions make this arrangement workable. The first concerns thresholds. A confidence of 0.8 returned by the model is only as good as the measurement behind it; you have to review a sample of decisions, count how often the model is wrong at each confidence level, and set the threshold for escalating to a person based on that count, not on the raw number. This is precisely what neither TypeSafe nor SemIf provides on your behalf. The second concerns the place of human validation. It does not disappear: it concentrates on the decisions whose consequences matter and on the ones the model itself flags as uncertain. The decision remains the process owner’s; what changes is that they no longer have to review hundreds of obvious cases to find the ten that deserve their attention.

And sometimes a simple rule is still better

All this sophistication can make us forget something obvious: some decisions need no model at all. Take a company where every invoice over 10,000 euros must be approved by a second person. A single line of code is enough: it compares the amount with the threshold and sends the invoice for a second approval. This rule is faster, more reliable and cheaper than adding a model to the process. It can be read and understood, and it always gives the same answer for the same invoice, whereas a model can answer differently at each iteration.

For each part of the problem, it uses the simplest mechanism capable of solving it correctly, and it only moves up a level when the previous one cannot reach a conclusion:

This is not as spectacular as an autonomous agent handed the entire problem. It is more reliable: each layer can be tested separately, and each layer feeds the one below it the data that will later let it handle more cases on its own.

After AI that produces, AI that chooses

For a few years, progress in models was mostly visible in what they produced: text, code, images, voice. Jev, and open experiments like SemIf, point to another axis. A model that does nothing but choose, and that chooses fast enough to start over as the world changes, enters loops where you observe, where you compare several actions, where you act, where you measure what actually happened.

What is still left to establish is just as clear. Jev’s figures come from its own vendor, its calibration has not been shown, and the open replicas document scores that are sensitive to the order of the options. We already use SemIf in our new automations, which is why we set its thresholds from reviewed decisions and route the uncertain cases to a person.

What this movement already tells us about automation seems solid.

Sometimes that means a rule, sometimes a tree-based model, sometimes a model able to understand a situation and decide in a hundred milliseconds. And when the problem calls for thinking, inventing, or explaining, only then is it time to wake up the large model.

If you have a process where a decision comes up dozens of times a day and today relies on a person reading through it, a thirty-minute conversation is usually enough to work out which of these layers fits it. Write to us at the bottom of the page, or book a slot directly in our calendar.

Sources

• TypeSafe AI, Introducing System One Models & Jev (15 September 2026)
  Official announcement: the latencies of 70 to 500 ms, the Doom demo running at about ten decisions per second, and its compute cost.

• TypeSafe AI, documentation: confidence
  Page describing the confidence score, which refers the user to their own measurements for calibration.

• Theodore Lee, SemIf (formerly OpenJev), GitHub repository
  Open replica introduced in the first part; its uncalibrated scores are the reason for the measured thresholds this article discusses.

• Hacker News, discussion of the TypeSafe AI launch
  Several developers point out that the Doom demo reads structured game state, not images.

• Chen, Zaharia, Zou, FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performance (2023)
  Model cascades: the accuracy of the best model for a fraction of the cost.

• Zhang et al., Generative Verifiers: Reward Modeling as Next-Token Prediction (ICLR 2025)
  Comparison between a judge that writes out its reasoning and a fast scorer in a selection loop.

• Hao et al., Reasoning with Language Model is Planning with World Model (EMNLP 2023)
  Tree-based exploration of decisions with a language model as evaluator.