The same checkpoint at two bits

What shipped

NVIDIA added two models to the Nemotron 3.5 family on 15 October 2026, with a third following later in the month. All three carry the OpenMDW-1.1 licence and are published on Hugging Face.

The architecture is inherited from Nemotron 3 Super, whose published configuration is a hybrid backbone that interleaves Mamba-2, attention and mixture-of-experts blocks: 88 layers, model dimension 4,096, head dimension 128, grouped-query attention with 32 query heads against just 2 key-value heads, and 512 routed experts with 22 selected per token. Hold on to the two key-value heads — they come back later.

NVIDIA's own throughput claim for the family is around five times faster on agent tasks than rivals, and press coverage of Super VL reports it beating named frontier models on specialised task benchmarks. Both are vendor-sourced figures on vendor-chosen tasks, and neither is the number this post is about.

The load-bearing phrase is "compressed from"

A derivative has a known ceiling and an unknown floor

Most of the open-weight models people put on single GPUs are trained at the size they ship. A 30-billion-parameter model is a 30-billion-parameter model: its capability is whatever its own training produced, and you can reason about it from its own evals. Bolt is a different object. Its capability ceiling is Super VL's, because compression removes precision and adds no training — whatever Super VL cannot do, Bolt cannot do. What is unknown is how far below that ceiling it lands, and that gap is a property of the compression, not of the architecture or the data.

This is a better position than it sounds for anyone who trusts the parent. You get a frontier-adjacent multimodal model's knowledge in a consumer memory budget, and you can read the parent's evals as an upper bound. It is also a worse position than a parameter count suggests, because the quantity you most need — the size of the gap — is the one the release does not quantify.

Native 2-bit is a different claim from 2-bit, and the difference matters

The field's rough consensus has been that below about 4 bits per weight, degradation stops being subtle. That consensus is about post-hoc quantisation: take a model trained at 16 bits, round its weights down, accept the damage. "Native" 2-bit implies something else — that the compression was part of how the checkpoint was produced, rather than applied to a finished model afterwards. Quantisation-aware training genuinely moves the floor, and a 2-bit model built that way is not comparable to a 2-bit round-down of the same weights.

That is the distinction a self-hoster needs resolved, and the word "native" gestures at it without settling it. Until the method is documented precisely enough to tell which regime you are in, treat the 2-bit claim as a hypothesis about quality that your own evaluation has to test — the same posture distillation and quantization recommends for any compressed build, applied with more force because the compression here is deeper than the usual default.

Why this matters more for an agent than for a chatbot

Quantisation damage is usually reported the way quality is usually reported — as a score on a benchmark where the model answers once. For an agent, that is the wrong denominator. A twenty-step loop needs every step to work, so per-step reliability compounds: 99% per step leaves 81.8% of tasks succeeding after twenty steps, 97% leaves 54.4%, and 95% leaves 35.9%. The two-point drop between the first two is a twenty-seven-point drop in finished work.

Compression does not damage all capabilities equally, and the ones it tends to damage are the agent-shaped ones: exact adherence to a tool schema, long-horizon goal retention, not repeating a call that already failed. These are precisely the behaviours that a single-turn benchmark either does not test or tests once, where a two-point difference looks like noise. So the model that benchmarks within a point of its parent can still be the model whose agent finishes half as many tasks, and nothing in either model card would tell you.

None of which is an argument against 2-bit local models. It is an argument about what you have to measure, and about reading task horizon rather than a leaderboard position when the deployment is a loop. Our page on small and local models makes the same point from the other direction: long-horizon tool use is where local models fall apart, and it is not close.

The 24 GB is the weights

74 billion parameters at two bits is roughly 18.5 GB of weights, which on a 24 GB card leaves something under 4 GB for everything else — and the attention cache is part of everything else. Read naively, a 256K context and a 24 GB budget do not fit together: on the geometry conventional for a 70-billion-parameter dense transformer, a 256K cache costs about 80 GB, more than three times the whole card.

Two architectural choices close that gap. Grouped-query attention with 2 key-value heads instead of 32 cuts the cache sixteen-fold against full multi-head attention over the same geometry. And in a hybrid backbone only the attention layers hold a cache that grows with length — Mamba blocks carry a fixed-size recurrent state instead — so the length-dependent cost is paid on a minority of the 88 layers. Divide the leftover budget by 256K tokens and you get an implied per-token cost of roughly 16 KB, which with two key-value heads at int8 implies on the order of 32 attention layers. That is a reconstruction from the published budget rather than a quote from a config file; the method, not the number, is the part to keep. Our essay on KV cache budgeting for agent loops works the arithmetic through properly.

The charge for that geometry is real and it lands in the same place the quantisation charge lands. A fixed-size recurrent state is lossy by construction, so a hybrid's advertised window and its usable recall can diverge for a structural reason rather than a benchmarking one — see effective versus advertised context. A model whose 256K is cheap in memory has to be paying for it somewhere, and "somewhere" is how much of that window it can still use at step 40.

What to measure before you adopt it

The release makes a specific comparison newly necessary, because three very different models now occupy the same slot. 74B at 2 bits, roughly 34B at 4 bits and roughly 12B at 8 bits are all about 24 GB of weights, and they fail differently: the first keeps breadth and risks precision, the last keeps precision and loses breadth. Equal-parameter comparisons no longer sort them; equal-memory comparisons do.

• Fix the memory budget, then compare. Pick the card you actually own, subtract the weights, subtract a cache sized for the context cap you will enforce, and shortlist only candidates that leave the same headroom. A model that fits only if you cap context at 16K is a different product from one that fits at 128K.
• Score per-step, not per-task. Instrument tool-call validity, schema adherence and repeat-failure rate per step across your own traces. One number — per-step success — predicts the loop, and raising it beats every other optimisation on the list.
• Run the eval at your real step count. A fifty-task suite at three steps per task tells you nothing about step forty. If your agent runs long, the eval has to run long, which costs more and is the cost worth paying.
• Compare the quantised build against its own parent, not against a leaderboard. The parent is the ceiling and the only honest baseline for measuring compression damage. If you cannot run Super VL locally, run it through a hosted endpoint for the eval alone.
• Treat the stated context as a budget, not a capability. Measure recall at the length you will actually reach, and set a context cap you enforce. Compaction is what keeps a local agent inside the memory you measured.

The broader pattern is worth naming, because more releases will look like this one. Vendors are shifting from publishing capabilities to publishing budgets — a memory figure, a context figure, a latency figure — and a budget is a claim about where a model fits rather than what it does inside it. That is useful information and it is not a substitute for the measurement. When a release leads with a number you can check by division, do the division; and when it leads with a number only your own traces can settle, go and get the traces.

Further reading

On this wiki:

• KV cache budgeting for agent loops — the arithmetic behind the 24 GB chart, and the capacity division it implies.
• Small and local models — which jobs never needed the big model, and where local models still lose badly.
• Distillation and quantization — what compression removes, and how to measure what it removed.
• Mixture of experts — why 9B active out of 74B buys compute and not memory.
• On-device agent architecture — the rest of the local stack, including cache warmth across restarts.
• Task horizon — why per-step reliability is the number that predicts a loop.
• Self-hosted inference for agents — the operational commitment you take on with the hardware.

Sources:

• Nemotron 3.5 Bolt 74B-A9B-24GB model card
• Nemotron 3.5 Super VL 120B-A12B BF16 model card
• NVIDIA: open source models and intelligent agents