Langfuse vs LangSmith vs Phoenix vs Braintrust: The Meter Is the Product

At a glance

Four platforms, sorted by what each one is genuinely built around rather than by what each one supports.

The trace archive is the asset, and it is the thing being metered

An agent trace is not a log line. It is a full record of a decision — what was retrieved, which tools were called with what arguments, what came back, what the model did next — and it feeds four downstream consumers that all matter more than the debugging view you bought it for. Your evaluation golden set is curated out of real traces. Your regression baseline is a pinned set of them. Your few-shot examples and any fine-tuning data are extracted from them. And your incident evidence, when a customer complains six weeks later, is whichever ones you still have.

That makes the archive the most valuable proprietary dataset most agent teams produce, and it is exactly what every commercial plan meters. Langfuse counts observations, LangSmith counts traces and seats, Braintrust counts scores and processed data volume. Each meter is defensible on its own terms, and each one prices the behaviour that makes agent debugging work: recording every step rather than the top-level call, letting the whole team look, and scoring runs continuously rather than in batches.

The predictable outcome is sampling. A team hits a plan ceiling, drops to tracing one run in a hundred, and the archive quietly stops being an evidence store — a change nobody notices until an investigation needs the run that was not kept, which is the argument made at length in retention and legal hold. Before comparing prices, work out your real volume: steps per task times tasks per day, not requests per day. An agent doing twenty steps produces twenty times the billable units of a chatbot serving the same number of users, and that factor is what turns a comfortable plan into an unexpected line item.

Where each one is genuinely strongest

Langfuse: the archive stays yours, and the licence says so

Langfuse is MIT licensed with self-hosting as a first-class path and no feature gates on the open-source build — tracing, evals, prompt management, datasets, experiments and the playground are all in it. That is a stronger commitment than "open core", and it is the reason it is the default recommendation for teams whose objection to a SaaS is the data rather than the price.

ClickHouse acquired Langfuse in January 2026, alongside a $400M Series D that valued ClickHouse at $15 billion, and the licence and self-hosting posture were explicitly kept. The technical fit was already there — Langfuse v3 ran on ClickHouse before the acquisition — and the practical read is that the analytical backend is now maintained by the company that makes it. The risk to weigh is the usual one after an acquisition: the roadmap now serves a larger strategy, and the parts of it that matter to a self-hoster are the parts to watch.

LangSmith: unmatched inside the stack it was built for

If your agent is LangGraph, LangSmith sees things a generic tracer cannot — graph state, node boundaries, the retry semantics of the framework itself — because it was built alongside them. That is a real advantage and it is worth choosing for, on the same reasoning that makes any first-party tool worth choosing.

The costs are two, and neither is hidden. It is closed source, with self-hosting reserved for enterprise plans, so the archive lives on somebody else's infrastructure until you are large enough to negotiate. And the metering is per trace with seats on top — listed at $39 per seat per month on the Plus plan with included traces per seat, against a free tier of a few thousand traces — which prices exactly the two things you want to grow: runs recorded and people looking at them. It also couples you to one framework's fortunes, which is a smaller risk than it sounds if you are staying, and a large one if you are not.

Phoenix: the standards-first choice, with a licence footnote

Phoenix is built on OpenTelemetry and describes itself as vendor, language and framework agnostic, and Arize also maintains OpenInference — the instrumentation and semantic-convention layer that a good deal of the ecosystem now uses to emit LLM spans. Choosing Phoenix therefore does something the others do not: it makes your instrumentation an asset independent of the platform, because the same spans go anywhere that speaks OTel. The self-hosted build has no usage limits and no phone-home, which for teams in regulated environments is often the whole decision.

Two things to be precise about. The licence is Elastic Licence 2.0 — source-available, not OSI open source: you can run it, modify it and self-host it without limits, and you cannot offer it to third parties as a managed service. For almost every team that distinction is irrelevant; for a platform vendor it is decisive, and it is regularly misreported. Second, Arize's ML-observability heritage shows up as drift detection and embedding analysis that the LLM-native tools approximate more loosely, which is worth something if you also run classical models and nothing if you do not.

Braintrust: the only one where the eval is the product

Everyone else added evals to a tracing tool. Braintrust started from scores and hung the traces off them, and the difference shows in the workflow: experiments, comparisons across runs and gating a deploy on a score are the main path rather than a section of the UI. If your team already practises eval-driven development, this is the platform that does not fight you.

The metering is the thing to model carefully, because it is unusual. Listed pricing meters scores and processed data volume — a free Starter tier with a small allowance of processed data and scores, and Pro at $249 per month with larger allowances and per-unit overage above them — with model tokens billed separately when you route through their proxy. The awkward incentive is that running more scorers per trace is both the recommended practice and the thing that increases the bill, so decide your scoring density deliberately rather than discovering it in an invoice.

One caveat covering all four price points above: this category re-prices frequently, and the plan structures move more than the platforms do. Treat the numbers here as the shape of each meter, verify them against current pricing pages, and weight any comparison article — including this one — accordingly.

The axis nobody sells on: instrument once, swap the backend

Here is the decision that outlives the choice you make today. Instrumentation is the expensive part — the spans, the attributes, the discipline of recording tool arguments and retrieval results consistently — and it is the part you do not want to redo. If you instrument against a vendor SDK, that work belongs to the vendor, and switching means touching the application. If you instrument against OpenTelemetry with OpenInference conventions, the application emits standard spans and knows nothing about who is receiving them.

Three consequences follow, and they are the practical core of this piece.

Send the same stream to two places from the start. One exporter to the platform, one to storage you control — object storage is fine. It costs an afternoon and it means the archive, the golden set and any future fine-tuning extract are yours regardless of what happens to the vendor relationship. This is also the only version of the retention question you can actually answer for a regulator.

Evaluate the migration before you need it. The question to ask a vendor is not "do you support OpenTelemetry" — everyone does now — but "can I export my historical traces in a form another platform can ingest". Answers differ sharply, and the difference only becomes visible at the moment you are least able to act on it. The vendor-risk framing in third-party and vendor risk applies here more than most people expect.

Standard semantics are what make evals portable, not just traces. A golden set built from spans that only one platform can parse is a golden set that migrates by rewrite. Following the OpenTelemetry GenAI conventions where they exist costs nothing at instrumentation time and is the difference between a switch and a project.

None of this argues against paying for a hosted platform. The UI, the search over traces, the eval tooling and not operating a ClickHouse cluster are worth real money. It argues that the payment should buy convenience rather than custody, and that the two are easy to conflate at the point of signing.

When to pick which

The recommendation underneath all six rows is the same: whichever you pick, instrument to OpenTelemetry and dual-write the stream somewhere you own. That single decision is worth more than the choice between these four, and it is the only one that is expensive to change later.

Further reading

On this wiki:

• Tracing & Observability — what an agent trace has to contain to be useful.
• OpenTelemetry GenAI Semantic Conventions — the standard that makes instrumentation portable.
• Eval-Driven Development & CI — the workflow Braintrust is built around.
• Retention & Legal Hold — why the sampling rate is a legal decision.
• Agent Observability — the concept, if you are starting from scratch.
• Promptfoo vs DeepEval vs Inspect AI — the eval frameworks that sit upstream of these platforms.

Project sources:

• Langfuse — GitHub
• ClickHouse welcomes Langfuse — ClickHouse blog
• Arize Phoenix — GitHub
• OpenInference — GitHub
• LangSmith documentation
• Braintrust documentation