Logits

In short

Logits are the raw scores a neural network gives each possible answer before they are turned into probabilities. In a chatbot, every word piece in the vocabulary gets a logit, and the highest scores are the likeliest next words.

Also called: logit, raw scores, unnormalised scores
Topic: How AI models work
Level: Some background helps
First described: Joseph Berkson introduced the term "logit" for the log of the odds in a 1944 paper in the Journal of the American Statistical Association, following Chester Bliss's "probit"⁵⁶.

When a language model picks its next token, it does not produce probabilities directly. Its final layer outputs one score per token in its vocabulary. Hugging Face's documentation describes these as "scores for each vocabulary token before SoftMax"¹.

Logits can be any number, positive or negative, and they do not add up to anything in particular. PyTorch notes they "do not need to be positive or sum to 1"². A function called softmax then converts them into probabilities that do sum to 1.

Many controls you meet in AI tools work on logits. Temperature rescales them: a value below 1 makes output less random, and a value near 0 puts almost all the probability on the single most likely token³. OpenAI's API has a logit_bias setting that adds a number, from -100 to 100, to chosen tokens' logits before sampling; -100 should in effect ban a token and 100 force it⁴.

Logits also matter in training. The standard cross-entropy loss in PyTorch takes logits as input and compares them with the right answer².

The word comes from statistics, where the logit of a probability is the log of its odds. Joseph Berkson coined it in 1944⁵⁶. In machine learning the term is used more loosely for any raw score fed into softmax².

Example

Ask a chatbot to finish "A cup of". Inside the model, "tea" and "coffee" might get high logits and "gravel" a very low one. Softmax turns those into probabilities, and the sampler picks one. Turning the temperature down makes "tea" or "coffee" even more certain to win³.

Why it matters

Logits are where a model's choices are actually made, so settings like temperature and banned words act on them. Knowing that explains why the same prompt can give different answers.

Questions people ask

What is the difference between logits and probabilities?

Logits are raw scores that can be any size, positive or negative, and need not sum to 1². Probabilities are what you get after passing logits through softmax: numbers between 0 and 1 that add up to 1.

What does logit bias do?

In OpenAI's API, logit_bias adds a number from -100 to 100 to the logits of tokens you choose before the model samples. Small values nudge a token's chances up or down; -100 or 100 should ban or force it⁴.

How does temperature relate to logits?

Temperature rescales the logits before they become probabilities. Values below 1 make the output less random, and a value of 0 is equivalent to always choosing the most likely token³.

Related terms

Sources

Back to the glossary