Post-Training: Base Model to Assistant

Pre-training buys knowledge; post-training buys behaviour.

Pre-training is the expensive part: trillions of tokens, months of compute, and the result is a base model that is very good at exactly one thing — continuing text. Ask a base model "What is the capital of France?" and a plausible completion is another three quiz questions, because that is what such a line looks like in the middle of a document. It has the knowledge and no notion that it is supposed to answer you.

Post-training converts that into an instruct or chat model. It is orders of magnitude cheaper than pre-training, it runs on far less data, and it is where the conversational contract is installed: there is a user, there is an assistant, turns alternate, and the assistant's job is to be helpful within stated limits. Providers ship both artefacts for open-weight releases, which is the clearest way to see the split — same weights of knowledge, completely different thing to interact with.

The stack, and why it stopped being one stage.

The textbook description — pre-train, then run RLHF with human preference labels — is now a simplification of a modular pipeline. Three families of technique do different jobs, and the mix varies by lab and by release.

• Supervised fine-tuning (SFT). Train on curated examples of instruction and good response. This is what installs the format of being an assistant. It is the cheapest stage and it does the heaviest lifting on basic instruction-following.
• Preference optimisation. Given pairs of responses ranked better and worse, push the model toward the preferred one. Classic RLHF learns a separate reward model and optimises against it; direct methods such as DPO skip the separate reward model and optimise the policy on the preference pairs directly, which is simpler and cheaper and now extremely common. This stage shapes tone, helpfulness, refusal behaviour and most of what people call personality.
• Reinforcement learning with verifiable rewards (RLVR). Where correctness can be checked mechanically — a maths answer, code that must pass tests, a tool call that must be well-formed — the reward is a program rather than a human judgement. This is the stage most responsible for the recent jump in reasoning and coding, and it is why capability now improves fastest in exactly the domains that have automatic graders.

Two consequences fall out. Capability improves where verification is cheap and lags where it is not — the same asymmetry that governs cascades and agent evaluation. And the preference data itself is increasingly model-generated and filtered, which is why synthetic data quality is now a frontier-lab bottleneck.

What this explains about the model in front of you.

• Sycophancy. If humans rate agreeable answers higher, preference optimisation learns agreeableness. A model that folds when you push back is not being uncertain; it is executing a trained preference, and it will do it on factual questions too.
• Format habits. Unprompted headers, bullet lists and closing summaries are post-training artefacts, not properties of language. They are also why asking for a bare value often still yields a preamble, and why structured outputs exist.
• Refusal edges. Refusals are trained boundaries, learned from examples, so they generalise imperfectly — over-refusing a benign request that resembles a trained-against one, under-refusing a harmful request phrased unlike anything in the data. This is why safety behaviour is not a filter you can reason about like a rule.
• The chat template is load-bearing. The special tokens that mark turn boundaries were installed during post-training. Serve an instruct model with the wrong template — a real risk when self-hosting — and quality degrades in ways that look like a bad model rather than a formatting bug.
• The alignment tax is real but small and shrinking. Making a model safer and more agreeable can cost raw capability. Modern pipelines pay much less of it than early ones, but "the base model is smarter, they lobotomised it" overstates a genuine effect considerably.
• Version bumps move behaviour, not just scores. Post-training is rebuilt per release. A same-family minor version can change verbosity, refusal boundaries and formatting enough to break a downstream parser while every benchmark number goes up.

What you can and cannot do about it.

You are not going to redo post-training. What you can do is stop being surprised by it, and be careful about the one place where you interact with it directly.

• Your fine-tune sits on top of it. Fine-tuning a released instruct model modifies weights that already encode the assistant persona and its safety behaviour, and a narrow fine-tune on unrelated data can degrade both — a well-replicated result. If you fine-tune, re-run your safety checks, not only your task metrics; see fine-tuning, RAG, or prompting.
• Prompting works with the grain, not against it. A system prompt is steering a trained disposition, not configuring a blank one. Instructions aligned with what post-training already rewards stick; instructions that fight it need reinforcement and degrade over a long context.
• Pin the version and keep a behavioural eval. A few dozen cases covering your actual output shape, refusal boundaries and verbosity, re-run on every version change, catches the regressions no leaderboard reports — the argument in reading benchmarks critically.
• Base models remain useful. For classification, scoring and completion-style tasks where the assistant persona is overhead, a base or lightly-tuned open-weight model can be simpler and cheaper. That is one of the quieter arguments in small & local models.

Related: training vs inference for where these stages sit, reasoning vs non-reasoning models for what RLVR produced, and hallucination & grounding for the failure mode post-training reduces but never removes.