Meta's new open-weights agent model scores 76.0 on SWE-Bench Verified and 24% on τ³-Banking, and both numbers are filed under "agentic". The gap is not a bad harness or an unlucky run — it separates an agent working alone against a test suite from an agent working with a person against a written policy, and only the second one describes the always-on local assistant this model was built for. Read the low number before you plan a laptop deployment.
What shipped
Meta Superintelligence Labs released Muse Glimmer in the second week of August 2026: 30 billion parameters, Apache 2.0, distilled down from a larger internal Muse system and shaped specifically for agent loops rather than chat. It is the company's most consequential open-weights release in a while, and the interesting part is not the licence.
| Property | Muse Glimmer 30B | Why it matters for an agent |
|---|---|---|
| Shape | 2B ViT-style perception encoder feeding a 28B text decoder; dense, not a mixture of experts. | Dense means memory tracks the full parameter count, which is the right trade when you own the GPU rather than rent it. See mixture of experts. |
| Context | 128K tokens. | Enough for a long tool loop, not enough to stop budgeting it. |
| Deployment | Compressed to roughly 4-bit, with block-level speculative decoding; runs on one consumer GPU or a Mac, no network call. | Both choices buy latency, not quality — see below. |
| Licence | Apache 2.0, weights on Hugging Face. | No acceptable-use rider, no user-count trigger. You can ship it inside a product. |
The headline scores are strong and were reported across a spread of agentic evaluations rather than the usual chat leaderboards. That spread is what makes the release worth reading carefully, because the scores do not agree with each other.
Five of these measure the same thing
AIME, SWE-Bench Verified, SWE-Bench Pro, MCP-Atlas and DeepSearch QA look like five different capabilities. Structurally they are one: a model alone in an environment, pursuing a goal that a program can check. Nobody interrupts. Nobody changes their mind at turn four. There is no rulebook the model is expected to have read and to keep obeying while being talked out of it.
τ³-Bench is the exception on the list. It is the third generation of Sierra's tool-agent-user benchmark, and its whole design point is to put a second party in the loop: the agent talks to a simulated customer, calls domain APIs, and is scored against a policy manual it must follow throughout. Policy adherence is a first-class metric, not a footnote, and the leaderboard reports pass^k — success on the same task k times running — rather than best-of-k. Banking is its newest and hardest domain.
That difference does real work. On SWE-Bench a failure is a red test — loud, immediate, and cheap. On τ³-Banking a failure can be a customer who hangs up happy after the agent waived a fee it had no authority to waive. The first kind of failure your CI catches. The second kind you find in an audit, or you do not find at all. It is the same asymmetry described in trajectory and process evaluation: outcome-only scoring cannot see a rule broken on the way to a good outcome.
The 24 is not Meta's problem alone
The reflex on seeing 24% is to conclude Meta cut a corner. The comparison points argue otherwise. In the same reported bracket, Gemini 3.5 Flash-Lite sits at 18% and Qwen3.6 27B at 17% — Muse Glimmer is ahead of its class, and the class is at the bottom of the chart. Meanwhile the frontier of τ³-Bench overall runs in the low seventies, and even those systems are reported to hold up in retail and airline while dropping sharply in banking.
So there are two effects stacked, and they should not be conflated. Banking is hard for everyone: more rules, more of them conditional, more places where the helpful action and the permitted action diverge. And small models are disproportionately bad at exactly this, because holding a long policy document in working attention across a dozen conversational turns while a user pushes back is a long-context discipline problem, not a knowledge problem — the effective-context failure described in effective context vs advertised, arriving under adversarial social pressure.
Which means the number to watch as local models improve is not SWE-Bench. Coding scores have been climbing on distillation alone. Policy adherence under multi-turn pressure has not moved nearly as fast, and it is the axis that gates every consumer assistant that touches money, health records or someone else's account.
What the engineering tells you Meta believes
Two shipping decisions are more informative than the benchmark table. Muse Glimmer is quantized to roughly 4 bits and it carries block-level speculative decoding. Neither improves quality; quantization costs a little and speculative decoding is provably output-neutral. Both buy per-token latency.
That is the correct obsession for an agent model and an unusual one to make this visible. A chat model pays its latency once per answer. An agent pays it once per step, and a modest task is ten to thirty steps, so a 200-millisecond difference in time-to-first-token becomes seconds of wall clock the user watches. Meta optimised the multiplier, not the multiplicand. Speculative decoding also has a known catch worth remembering before you assume the laptop numbers transfer: it trades extra compute for lower latency, so it wins on an idle personal GPU and quietly loses throughput on a busy shared one.
The 2B perception encoder points the same way. On-device agents spend their time reading screens, receipts and PDFs, not describing photographs. A small encoder bolted to a large decoder is a bet that the vision work is mostly transcription.
Where a 30B local agent actually earns its keep
Split the work by whether a rule binds the outcome. Classification, extraction, drafting, summarising, routing and the read half of any tool loop are all things a 30B model does well and a local one does instantly and free — that is the case small and local models has always made, and Muse Glimmer strengthens it. Anything that commits the user to a policy, a payment or another person goes over the escalation gate to a frontier model, or to a person. Same architecture as a cascade, with the router being a static rule rather than a difficulty estimate; see model routing and cascades for why static usually beats clever here.
Two roles are underrated and this model is well shaped for both. It is a good local judge for cheap pre-filtering before you spend a frontier call on evaluation, and it is a good local pre-classifier that decides whether a request needs the network at all — which is the single highest-leverage thing you can put in front of a metered API.
And the caveat that gets skipped in every "runs on your device" launch: a local model is not a local system. The moment your agent calls a calendar, a mail provider or a payment API, the user's data crosses the boundary in a request the model never sees the other side of. Running inference locally removes one egress path and leaves the rest exactly where they were. That is a configuration problem, and it is the subject of egress control for agents, not a property you get from the weights.
FAQ
Is Muse Glimmer good enough to replace a frontier model in my agent?
For the code-shaped and retrieval-shaped parts of the loop, plausibly yes — the reported SWE-Bench Verified and DeepSearch QA numbers are in serious territory for 30B. For multi-turn work bound by a written policy, the reported τ³-Banking score says no, and no prompt will close a gap of that size.
Why does a model score 94.7 on AIME and 24 on a customer-service benchmark?
Because they are not the same skill. AIME is a single hard question with a checkable answer and no second party. τ³-Banking is a dozen easy turns in which the agent must keep obeying a rulebook while a simulated customer pushes for an exception. Reasoning depth does not buy policy discipline.
Does Apache 2.0 actually matter here?
It matters more than the benchmark spread for anyone shipping a product. A permissive licence with no acceptable-use rider and no user-count trigger means the weights can be embedded, fine-tuned and redistributed without a legal review per release, which is the practical difference between an experiment and a dependency.
What should I measure before trusting a local agent model?
Your own policy-adherence rate over multi-turn transcripts, scored pass^k rather than best-of-k, on the rules you actually enforce. A published benchmark tells you the class of model is capable; only your suite tells you the model holds your rules. See reading agent benchmarks.
Further reading
On this wiki:
- On-device agent architecture — how to build the hybrid split above without pretending the device boundary is a security boundary.
- Task horizon — why the reliability number is roughly five times shorter than the capability number everyone quotes.
- Small and local models — the standing case for running a small model, and where it stops.
- Prefill, decode and the KV cache — why latency, not parameter count, is the binding constraint in a loop.
- Self-hosted inference for agents — what changes operationally when you own the GPU.
Sources:
- Meta AI Research — Introducing Muse Glimmer
- sierra-research/tau2-bench — the repository that maintains τ³-Bench
- Artificial Analysis — Muse Glimmer benchmarks