Training data

In short

Training data is the collection of examples an AI model learns from, such as text, images or labelled records. The model adjusts its internal numbers to fit patterns in this data, so what goes in largely shapes what comes out.

Also called: Training set, Training dataset
Topic: The basics
Level: Beginner

The EU AI Act defines training data as "data used for training an AI system through fitting its learnable parameters"¹. It keeps this separate from validation data, which is used to evaluate the trained system and tune its other settings¹.

For chatbots the scale is vast. Most of GPT-3's training text came from Common Crawl, an archive of web pages; OpenAI filtered 45TB of compressed text down to 570GB². Meta trained Llama 2 on 2 trillion tokens from publicly available sources, and said it tried to remove sites known to hold a lot of personal information about private individuals³.

Image generators learn from pictures. In Getty Images v Stability AI, the High Court heard that training Stable Diffusion meant downloading images from web links in the LAION datasets onto Amazon cloud servers⁴. Getty dropped its claim about training after it was accepted that training did not take place in the UK⁴.

Training data is not kept inside the finished model as a library. The same judgment recorded that the model "does not store training data"⁴. But the data still shapes the model. The US standards body NIST warns that harmful bias in generative AI "may stem from their training data"⁵.

Example

A model for spotting fractures on X-rays would be trained on thousands of scans that doctors have already labelled. The labels are what it learns to predict when it sees a new scan.

Why it matters

Training data decides what a model knows and gets wrong, and who owns that data is one of the biggest legal fights in AI, including in UK courts⁴.

Questions people ask

Where does AI training data come from?

Much of it is collected from the public web. GPT-3's largest source was a filtered copy of Common Crawl², and Llama 2 used 2 trillion tokens from publicly available sources³.

Is it legal to train AI on copyrighted work in the UK?

It is not settled by the courts. In Getty Images v Stability AI (2025) the claim about training was dropped because training did not happen in the UK, so the High Court did not rule on it⁴.

Does a model keep a copy of its training data?

Not as a stored library. The High Court found that Stable Diffusion's model "does not store training data"⁴. What it keeps are learned parameters shaped by that data¹.

Related terms

Sources

Back to the glossary