AI Blog

Tagged: self-hosted

← Back to AI Blog

10 min read

Together vs Fireworks vs Baseten vs Modal: Agents Break Per-Token Pricing

A chat product needs several hundred concurrent users before a dedicated GPU beats per-token pricing; an agent needs about a dozen workers, because it re-sends its whole context every step. That arithmetic — not the price per million tokens — is what should decide which of these four you build on.

7 min read

Muse Glimmer Ships Two Agentic Numbers, and the Wrong One Is in the Headline

Meta's 30B open-weights agent model scores 76.0 on SWE-Bench Verified and 24% on τ³-Banking. Five of its six headline numbers measure a model alone against a machine-checkable goal; the sixth measures it working with a person against a written policy — and that is the axis an always-on local assistant lives on.

9 min read

vLLM vs SGLang vs TensorRT-LLM vs llama.cpp: Throughput Is the Wrong Benchmark for Agents

Every comparison of these four opens with tokens per second on a fixed batch — the one number that transfers worst to agent traffic, where the same prompt comes back twenty times with a few hundred tokens appended. What separates them is what the KV cache is keyed on, whether constrained decoding survives a full batch, and how much of your quarter the build step eats.

11 min read

Docling vs Unstructured vs LlamaParse vs Mistral OCR: Stop Choosing a Parser on Accuracy

Every document-parser comparison is published as an accuracy leaderboard, and accuracy is the axis that transfers worst to your documents. Two things do transfer: a layout pipeline can drop a number but cannot invent one, and the cost curves of self-hosted and hosted parsing cross at a volume you can compute in five minutes.

9 min read

Kimi K3 Is Open Weights. That Is Not the Same as Cheap, Local, or Unrestricted

Moonshot released 2.8 trillion parameters as a free download on 27 July — and priced its own API above the model it replaced, while no single GPU on the market can hold the weights. Open weights buy agent builders exactly one thing that closed APIs cannot, and it is not cost.

14 min read

LanceDB vs Chroma vs sqlite-vec vs FAISS: Four Shapes for a Local Agent Knowledge Base

Before you pick a local vector store, notice that Claude Code, Cursor and Codex deleted theirs — the leading coding agents retrieve with grep, not embeddings. If your corpus still needs an index, these four are not competing products but four different architectures: a search library with no storage, a SQLite extension, an embedded engine with a write-ahead log, and a columnar format on disk.

17 min read

Llama 4 vs DeepSeek V3 vs Qwen3 vs Mistral Large 3: Four Open-Weights Flagships, Four Different Bets

Every few months, four labs ship a similar-sounding open-weights flagship — MoE, long context, reasoning mode, multimodal. The benchmarks keep getting passed back and forth. The thing that actually decides which one you run in production is the axis each lab is betting on next: multimodal ecosystem, inference economics, agentic reasoning, or permissive-license frontier intelligence.