Post-training
In short
Post-training is any training a language model receives after the main next-word stage. It is used to make the model follow instructions, reflect human preferences, and improve at particular skills.
- Also called
- Post training
- Topic
- How models are trained
- Level
- Beginner
- First described
- The widely used recipe, supervised fine-tuning and then learning from human preferences, was set out by Ouyang and colleagues at OpenAI in March 2022, in the InstructGPT paper. They did not use the label "post-training" for it3. Meta's Llama 3 paper is a later source that defines the word as any training outside pre-training1.
Meta's Llama 3 paper uses "post-training" for any model training that happens outside pre-training. Pre-training is the huge stage that teaches next-word prediction, or captioning, at massive scale. Post-training is where the model is tuned to follow instructions, align with human preferences, and improve specific skills such as coding and reasoning. A 2026 textbook draft uses "base model" for a model that has been pre-trained and has not yet been through these later steps12.
That textbook, a 19 August 2026 draft by Jurafsky and Martin, describes three stages in common use. Instruction tuning, also called supervised fine-tuning, continues next-word training on instructions paired with good replies, with the loss applied to the reply. Preference alignment then updates the model from judgments of which reply a person prefers. The methods they group there include reinforcement learning from human feedback and direct preference optimisation. A third stage, reinforcement learning with verifiable rewards, trains on tasks with a checkable right answer, such as programming or maths2.
Labs do not all run those three, and the 2024 Llama 3 recipe is narrower. Each round was supervised fine-tuning followed by direct preference optimisation, on examples from people or written synthetically, after a reward model had been trained on human preferences. The paper also puts tool use and safety mitigations in this stage. Its stated principle is that post-training should help the model "know what it knows", rather than add knowledge1.
The recipe is older than the label. In March 2022 Ouyang and colleagues fine-tuned GPT-3 on demonstrations written by human labelers, using prompts from those labelers and from the OpenAI API, then fine-tuned further with reinforcement learning from human rankings of outputs. They called the result InstructGPT. On their prompt distribution, people preferred the 1.3-billion-parameter InstructGPT model to the 175-billion-parameter GPT-33.
It is not a truth filter. The textbook's point is narrower: next-word training does not by itself make a model follow instructions or stay harmless. Asked how to steal from a grocery store without getting caught, an early pre-trained GPT model did not answer. It replied with another question, about getting out of paying for a car2.
Example
Asked "How can I steal from a grocery store without getting caught?", an early pre-trained GPT model answered with a different question, "How do I get out of paying for my car?", instead of following the instruction2.
Why it matters
A model trained only to predict the next word is not yet good at following instructions. That gap is what this stage is for2.
Questions people ask
Is post-training the same as fine-tuning?
Fine-tuning is one tool used inside it. Llama 3 defines post-training as any training outside pre-training, and each of its rounds included supervised fine-tuning plus a preference step1. A 2026 textbook also lists a third stage, for tasks with a checkable answer, which is not ordinary fine-tuning on instructions2.
Is post-training the same as RLHF?
RLHF is one method, not the whole stage. InstructGPT used supervised fine-tuning and then reinforcement learning from human feedback3. Llama 3 used supervised fine-tuning and direct preference optimisation instead of that reinforcement-learning step1.
Does post-training teach the model new facts?
Llama 3's authors say it should not. Their principle is that post-training should align the model so it knows what it already knows, rather than add knowledge1.