Grokking
In short
Grokking is when a neural network memorises its training examples and still fails new ones, then much later, after far more training, starts getting new examples of the same task right.
- Topic
- How models are trained
- Level
- Some background helps
- First described
- Alethea Power and colleagues, then at OpenAI, January 2022, in "Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets"1.
Power and colleagues, then at OpenAI, named this pattern in January 2022. They trained small transformers on tables of abstract symbols, equations of the form a ∘ b = c, where the symbols have no decimal spelling the model can lean on. The job is to fill in equations it was not shown. They note that the experiments can be reproduced on a single GPU, which makes them a testbed for ideas about generalisation1.
The case they highlight is division modulo 97, a wraparound arithmetic with 97 values, with 50% of the equations in the training set. Training accuracy was close to perfect in fewer than 1,000 optimisation steps. Accuracy on the held-out equations showed very little generalisation until about 100,000 steps, and reached the training level only at close to 1 million steps. That is about 1,000 times as many steps as memorisation took1.
That is sharper than the small late gains people usually see once training accuracy has saturated. It appeared across several operations, most clearly when the training set was only just large enough for the network to generalise within the step budget. As the training fraction shrank, the number of steps needed to reach 99% validation accuracy grew quickly. Of the tweaks they compared, weight decay, a penalty on large weights, improved generalisation the most1.
A later study of a related task, modular addition, offers a mechanism. Nanda and colleagues reverse-engineered a small transformer and report that it maps the inputs onto a circle and adds them as rotations, using sines and cosines at a few frequencies. They split training into memorisation, circuit formation, and cleanup. On their account grokking is not a sudden flip: structured machinery in the weights is amplified gradually, and the memorising parts are removed later2.
Example
A small network trained on division modulo 97, with half the sums held back, had the practised sums almost right in under 1,000 steps, but was still near guesswork on new sums at about 100,000 steps1.
Why it matters
A perfect score on the training set can arrive long before the network has a rule that works on new cases. Stopping when the training score looks done can mean stopping too early.
Questions people ask
Is grokking the same as overfitting?
No. Overfitting is fitting the training set so tightly that new examples suffer. In grokking, the network does overfit first: training accuracy is near perfect while validation accuracy is still near chance. The name is for what happens next, when validation accuracy later climbs toward perfect1.
Is grokking the same as an emergent ability?
No. Grokking, as Power and colleagues define it, is one small network trained far past memorisation on a tiny algorithmic dataset1. A jump that appears only as models get larger is a different claim, and this paper does not study it.