Weights
In short
Weights are the learned numbers inside an AI model that decide how strongly each input affects the output. Together they hold everything the model picked up in training, and they can be saved and shared as files.
- Also called
- Model weights, Neural network weights, Weight
- Topic
- The basics
- Level
- Beginner
- First described
- Rumelhart, Hinton and Williams set out how a network learns by repeatedly adjusting the weights of its connections, in Nature on 9 October 1986.
In a neural network, each connection between units carries a number. A unit adds up its inputs, each multiplied by the weight on its connection, and passes the result on1. Training changes those numbers. The 1986 backpropagation paper describes a procedure that "repeatedly adjusts the weights of the connections" to shrink the gap between the network's output and the right answer1.
A US government report uses a similar plain definition: model weights are "numerical parameters within an AI model that helps determine the model's output in response to inputs"2. The same report notes there are several kinds, including pre-trained weights, weights saved at checkpoints during training, and weights of fine-tuned models2.
Weights are the valuable part of a model. They are the product of an enormous amount of computing. Meta's largest Llama 3 model, for instance, took 3.8 × 10^25 floating point operations to pre-train3.
That is why the main dividing line in AI is who gets the weights. Open-weight models, such as Meta's Llama 3 or OpenAI's gpt-oss, let anyone download them34. Closed models keep them on the company's servers. The US report found that downloadable weights let people strip out built-in safety features more easily than with closed models, but did not recommend restricting them at the time2.
Example
When someone downloads a model such as gpt-oss-20b to run on their own computer, most of what they download is the weights. OpenAI says that model can run within 16GB of memory4.
Why it matters
Whoever holds a model's weights holds the model. Debates about open and closed AI, model theft and safety rules are really about who can access these numbers.
Questions people ask
Are weights the same as parameters?
Almost. Weights are most of a model's parameters. The rest are biases, which the 1986 backpropagation paper handled as just another weight1. Many people use the two terms interchangeably.
What does it mean to release model weights?
It means publishing the trained numbers so anyone can download and run the model on their own hardware. OpenAI, for example, released the weights of its gpt-oss models under the Apache 2.0 licence4.
Can you see what a model knows by looking at its weights?
Not directly. The weights are billions of plain numbers with no labels. Working out what they encode is the job of a research field called interpretability.
Related terms
Tools that use it
- Meta Llama modelsMeta's family of downloadable large language models, from Llama 2 to the multimodal Llama 4
- OllamaFree tool for running open AI models on a home computer, with optional paid cloud models
Sources
- Rumelhart, Hinton and Williams (1986), Learning representations by back-propagating errors, Nature 323, 533-536
- NTIA, US Department of Commerce (July 2024), Dual-Use Foundation Models with Widely Available Model Weights
- Grattafiori et al. (2024), The Llama 3 Herd of Models, arXiv:2407.21783
- OpenAI, gpt-oss-120b model card on Hugging Face (covers gpt-oss-120b and gpt-oss-20b)