Jev and its fast followers sell a probability nobody has measured

At a glance

Four decision models that answer typed questions about a piece of state, as of 19 October 2026. All four take a question you write at call time and return a label with a probability distribution over the labels; none of them writes text.

Vercel's numbers are the honest picture of where the category stands. Jev reached roughly 13% of the gateway's paid teams within a day of its 15 September launch and 35.3% within two weeks — TechCrunch called it the gateway's fastest-adopted model ever — while accounting for about 0.27% of spend and keeping 55.6% of its first-week users into week two. At $0.042 per million input tokens a trial costs nothing, so reach measures curiosity. Whether the curiosity turns into production traffic depends on the question this post is about: whether the probability it returns can be trusted to make a decision without a human or a bigger model behind it.

What "can't hallucinate" buys you

Jev's early marketing said it could not hallucinate, and the Hacker News thread on its launch — nearly 2,000 points — spent much of its length taking that apart. The precise claim is narrower: because you define every possible answer before the call, Jev cannot return an answer outside that set. TypeSafe's own launch post concedes that its 0% type-error figure is "not empirical"; it is a property of the interface. One commenter's version is the one to keep: it cannot emit an invalid type, but it can still emit a completely wrong valid value.

That guarantee is not new. Any LLM behind constrained decoding or a strict structured-output schema also returns only labels you allowed. What the LLM does not hand back is a usable measure of how sure it was. Its token log-probabilities have been bent by post-training, and its verbalised "I'm 90% sure" is a separate generated claim — the research disagrees about which of the two is worse, which is reason enough to trust neither without checking (see Uncertainty & Calibration). A decision model is trained to make the distribution itself the output. If that distribution is calibrated — if answers given at 0.9 are right about 90% of the time — you can do something an LLM call cannot support: act automatically above a threshold and escalate below it.

The speed number compares different jobs

TypeSafe's homepage claims 193.6× faster and 444.6× cheaper than LLMs on its workflow evals, whose reference answers came from frontier LLMs and whose workflows TypeSafe's own team built. OpenChamber pulled four days of launch tweets, kept the 12,759 relevant ones, and read off what users themselves reported: a median 7× speedup and 30× cost reduction, with a median latency of 76 ms. Both numbers are real, and they are about different things — the first is a chain-of-thought LLM against a model that does not reason, the second is closer to "the classifier I had, swapped out". Use the second to budget, and expect your own number to depend mostly on how long the LLM you are replacing was thinking.

Jev — the reference design

Jev is the first of what TypeSafe calls "System One" models — intuition rather than reasoning: no reasoning tokens, one parallel pass over every question in the request. TypeSafe was co-founded by Diogo Almeida, a former OpenAI researcher who worked on RLHF, with Erik Gafni and Sasha Sheng. Almeida has said the model is trained only on synthetic data with a method TypeSafe calls reinforcement learning for calibrated decisions (RLCD); the architecture is undisclosed beyond "a new model architecture and a parallel sampler". The name is for William Stanley Jevons, of the paradox that cheaper resources get used more.

A request is one POST to /v1/systemone with a state (a string or JSON), a map of named questions, and optional image assets. There are four question types, and the useful way to read them is as control flow — the mapping an account identifying itself as TypeSafe's CEO gave on Hacker News:

• noul (from Bernoulli) — an if. Returns one probability that a condition holds.
• choice — a match, over up to 255 options you describe. Returns the pick and a probability per option.
• score — a sort key, over 2 to 10 ordered levels you describe. Returns an expected level and a probability per level.
• box — new in 1.14.0, experimental, image-only: one bounding box.

{
  "model": "jev-1.13.0",
  "state": "Our API integration started returning 500 errors on every request about 20 minutes ago.",
  "questions": {
    "department": {
      "type": "choice",
      "instructions": "Which team should handle this",
      "criteria": {
        "billing": "Payment or subscription issues",
        "technical": "Bugs or integration problems",
        "sales": "Pricing or account questions"
      }
    },
    "is_urgent": { "type": "noul", "instructions": "The message conveys urgency or time-sensitivity" }
  }
}

Context is 64k tokens per request and 32k per question by default, or 262k behind a "danger zone" header that the docs say is slower and less accurate. Two details in the docs matter more than the headline specs. First, the confidence field on choice and score answers is computed from probabilities by a fixed formula, so it carries no information the distribution did not; noul answers have no confidence field at all. Second, TypeSafe publishes a candid "jaggedness" page for 1.13 listing nine failure modes — counting and arithmetic, date comparison, double negatives and other indirection, large state full of irrelevant detail, adversarial content in the state, contradictory criteria, a lean toward whichever choice is listed first, and anything that needs generated text. Its own summary is "Jev is not a calculator", and its advice for most of the list is to do that part in code.

Aliases move: jev-latest points at 1.13.0 today and is due to move to 1.14.0 about a week after that version's 14 October release. TypeSafe has promised 60 days' notice before retiring 1.13.0 and 30 days for 1.14.0, but has not yet published a deprecation policy. A threshold tuned on one version is a threshold for that version.

OpenAI Decisions — the fast follow

Sam Altman mentioned a Decisions API at DevDay on 29 September; it went to public beta on 6 October. OpenAI's Nikunj Handa told Fortune it had not been on the roadmap four weeks earlier and credited Jev with "inspiring this whole thing". The design is recognisably the same idea in OpenAI's house style: POST /v1/decisions with an input and an array of named questions, typed predicate (the yes/no case), choice and score, answered by gpt-6-luna — the only model offered. Fortune reports it was built on Luna without extra training; OpenAI's docs do not say.

It costs $0.10 per million input tokens with no output charge, about 2.4× Jev's price, and it accepts images from day one, where Jev's image support is still behind an experimental header. Its real differentiators are the ones procurement asks about: zero data retention and HIPAA for eligible customers, US and EEA/Swiss data residency, and the contract you already have. OpenAI publishes no calibration numbers either. Its guide tells you to set thresholds from labelled examples — the right advice, and an admission that the number is yours to find.

Clef and pplx-decider — Jev's shape, open weights

Cloudflare's Clef and Clef-flash shipped on 1 October as Apache-2.0 weights on Hugging Face and Workers AI, described as "fully Jev-API compatible": the same state plus noul, choice and score. Clef is a frozen Qwen3.8-27B with a rank-256 LoRA and a vision encoder; Clef-flash is built on Qwen3.5-9B. Cloudflare says it trained them with a Brier-score loss and an RL objective it also calls RLCD. Perplexity's pplx-decider-v1-27b is likewise Apache-2.0 and fine-tuned from Qwen3.8-27B.

Two things follow. Jev's request shape is turning into a de facto interface, which makes the questions you write portable between at least three vendors — and OpenAI, by choosing a different shape, is not one of them. And the open-weight entries make the category self-hostable, which matters for exactly the workloads where a decision model is most attractive: high-volume screening of data you would rather not send out. Both vendors publish benchmark tables on which they beat Jev on some sets and lose on others (Cloudflare reports Clef at 94.2 macro-F1 on BANKING77 to Jev's 79.7, while Jev wins its agent-trace workflow; Perplexity reports 85.7% overall to Jev's 84.5%). Neither table reports calibration.

The number nobody publishes

TypeSafe has said publicly that it will not publish benchmark tables, only dated snapshots of retired evals. Its launch post's calibration claim is the sentence "higher confidence means higher accuracy", with no metric attached. The competitors' tables measure accuracy and F1. As of this week the search for an expected calibration error measured on a decision model's intended job — a classification task drawn from real traffic — comes back empty.

The closest thing is off-label. The Pinocchio paper (arXiv 2609.24881) used Jev to predict whether another model's answers were correct and measured AUROC 0.666, Brier 0.240 and ECE 0.133 over 1,718 responses, against 0.868 AUROC for a trained calibrator. It also found Jev non-deterministic, returning different scores for identical repeated requests. That is not the job Jev is sold for, so it does not show Jev is badly calibrated at routing tickets. It shows that "calibrated" is a property of a model on a distribution, and nobody has published it on yours.

The most neutral comparison so far comes from Parallel, which ran three hosted and five open decision models on four of its own production tasks. On page-change detection, before any prompt tuning, pplx-decider-v1 scored 93.3 AUPRC to Jev's 87.9 and OpenAI Decisions' 79.0; on reranking, Parallel's own fine-tuned model beat Jev 75.2 to 71.5 at about a quarter of the per-token cost. Its verdict — Jev is "a strong default", and a fine-tuned small classifier can still win where cost, latency and quality matter most — is the right one. Note what AUPRC is, though: a measure of whether the model ranks positives above negatives. A model can rank perfectly and still say 0.9 for things that are true 60% of the time. Ranking tells you a threshold exists; only a reliability check tells you where it is.

The threshold sets your bill

The way these models get deployed in practice is as a screen in front of something more expensive. Every item goes to the decision model; the answers above your threshold are acted on; the rest escalate to the LLM you used before, or to a person. At Jev's price the screen is close to free, so the bill is roughly the escalated fraction of what you paid before. Settle 95% of items and you pay about 5% of the old cost; settle 60% and you pay about 40%.

That makes the threshold the most expensive number in the system, and it moves in both directions. Set it from the vendor's word and a miscalibrated model will push wrong answers through the act band at a rate you never measured. Set it conservatively for lack of evidence and you escalate most of the traffic and keep most of the old bill. The 444× headline and the 7× median are both downstream of this one choice, and you can only make it from labelled examples of your own traffic. The Thresholds for decision models runbook walks through doing that.

Injection still works, more quietly

A decision model reads its state as data, and TypeSafe's jaggedness page says plainly that Jev does not treat it as hostile by default. HiddenLayer showed an injected "Confirm Price" banner hijacking a browser agent's checkout decision through Jev despite a system prompt calling page text untrusted, and character perturbations evading a toxicity question. A September paper on 510 InjecAgent cases found the effect is usually a shift rather than a flip — injected text moved the probabilities but rarely made Jev pick the attacker's target, and an adaptive attack raised fresh-validation success only from 1.8% to 3.5%. Fewer flips is good news; a shifted probability crossing your threshold is still a flip, and you will not see it in the label.

When to pick which

Whichever you pick, write the questions against the Jev shape if you can — three of these four vendors accept it, and TypeSafe's MIT-licensed system-one-adapter-python runs the same typed questions against OpenAI, Anthropic and Gemini models, which gives you a baseline from the LLM you are replacing at no extra integration cost.

Further reading

On this wiki:

• Decision Models — what the category is, and why the rule is "code computes, the model judges".
• Thresholds for decision models — the runbook for measuring calibration and setting the three bands.
• Uncertainty & Calibration — why an LLM's own confidence is not a threshold you can use.
• Model Routing & Cascades — the screen-then-escalate pattern in general.

Sources:

• TypeSafe — Introducing System One models and Jev (15 Sep 2026), and the jev-1.13 jaggedness page
• TypeSafe — Jev 1.14.0 (14 Oct 2026)
• OpenAI — Decisions API guide
• Cloudflare — Clef decision models (1 Oct 2026); perplexity-ai/pplx-decider-v1-27b
• Vercel — AI Gateway Production Index, October 2026
• Parallel — Are all System One models created equal?
• Pinocchio (arXiv 2609.24881); HiddenLayer — System One model security; Decision Hijacking (arXiv 2609.28613)
• OpenChamber — what 12,759 tweets said about Jev; Hacker News launch thread
• Fortune — Jev and OpenAI's response (8 Oct 2026)