Chain of thought
In short
Chain of thought (CoT) is when an AI model writes out intermediate reasoning steps before its final answer, like showing your working in a maths exam. It can be asked for in a prompt or trained into the model.
- Also called
- CoT, Chain-of-thought prompting, Chain-of-thought reasoning
- Topic
- How AI models work
- Level
- Some background helps
- First described
- Jason Wei and colleagues, January 2022, in "Chain-of-Thought Prompting Elicits Reasoning in Large Language Models"1.
The idea was named in a January 2022 paper by Jason Wei and colleagues at Google Research. They called a chain of thought "a series of intermediate reasoning steps" and showed that giving a model a few worked examples with steps in the prompt improved arithmetic, commonsense and symbolic reasoning1.
The gains were large. A 540 billion parameter model prompted with just eight chain-of-thought examples reached state-of-the-art accuracy on GSM8K, a set of school maths word problems, beating a fine-tuned GPT-3 with a verifier1. The paper found the ability appears in sufficiently large models1.
A few months later Kojima and colleagues showed you do not even need examples. Adding "Let's think step by step" before the answer lifted a large InstructGPT model's accuracy on the MultiArith maths test from 17.7% to 78.7%, and on GSM8K from 10.4% to 40.7%2.
Today chain of thought is often built in rather than prompted. OpenAI's o1 models are trained with reinforcement learning to reason using chain of thought before they answer3. These are known as reasoning models.
A chain of thought is not a reliable window into the model's mind. Anthropic researchers tested whether reasoning models admitted to using hints planted in a prompt, and found they often did not, with the reveal rate often below 20%4. The researchers conclude that reading the chain of thought can catch some bad behaviour but cannot rule it out4.
Example
Asked "A bus fare is £2.50 and you make 7 journeys. How much change from £20?", a chain-of-thought answer writes "7 x £2.50 = £17.50, and £20 minus £17.50 = £2.50" before giving £2.50 as the answer.
Why it matters
It is one of the simplest ways to get better answers from a chatbot on multi-step problems, and it is the basis of today's reasoning models.
Questions people ask
Does "let's think step by step" really work?
It did in testing. Kojima and colleagues found that adding that one phrase raised a large InstructGPT model's score on the GSM8K maths test from 10.4% to 40.7%, with no worked examples needed2. Newer reasoning models already think step by step, so the phrase matters less with them.
Is chain of thought the same as a reasoning model?
No. Chain of thought is the technique of writing out steps. A reasoning model is one trained, with reinforcement learning, to produce a chain of thought on its own before answering3.
Does the chain of thought show what the AI is really thinking?
Not reliably. Tests on reasoning models found that when they used a hint in the prompt, they often did not say so, with the reveal rate often below 20%4.
Related terms
Tools that use it
- ChatGPTOpenAI's general chatbot, the most used AI app in the UK and worldwide.
- ClaudeAnthropic's chatbot, strong on writing and on pointing out holes in an argument, with a tight free allowance.
- DeepSeekA free Chinese chatbot with no message cap that stores every conversation on servers in China.
Sources
- Wei et al. (2022), Chain-of-Thought Prompting Elicits Reasoning in Large Language Models, arXiv:2201.11903
- Kojima et al. (2022), Large Language Models are Zero-Shot Reasoners, arXiv:2205.11916
- OpenAI (2024), OpenAI o1 System Card, arXiv:2412.16720
- Chen et al. (2025), Reasoning Models Don't Always Say What They Think, Anthropic (paper also at arXiv:2505.05410)