Evals
In short
Evals (short for evaluations) are structured tests that measure how an AI model or AI product behaves, from accuracy on a task to safety risks. Developers run them before and after releases to check a change made things better, not worse.
- Also called
- AI evals, Evaluations, Model evaluations, LLM evals
- Topic
- Testing and benchmarks
- Level
- Some background helps
- First described
- Evaluation has always been part of machine learning. OpenAI released its open-source Evals framework with GPT-4 in March 20231.
An eval is a set of inputs, a way of running the model on them, and a way of scoring the outputs. Public benchmarks such as MMLU are one kind. Many evals are private, built by a company around its own product.
OpenAI open-sourced its Evals framework alongside GPT-4 in March 2023, describing it as a tool for creating and running benchmarks and inspecting results sample by sample1. Its documentation says that without evals it is hard to know how a new model version will affect your use case2.
Scoring can be automatic, by matching an exact answer, by running tests on code, or by asking another model to grade the answer. A 2023 study found strong models used as judges agreed with human preferences over 80% of the time, about the same as humans agree with each other3.
Governments run evals too. The UK AI Security Institute built Inspect, an open-source framework for testing coding, agentic tasks, reasoning, knowledge and behaviour4. The research group METR moved its own testing onto Inspect in 2026, calling it widely adopted5.
Evals are not proof a system is safe or useful. A model can pass a test set and still fail on situations nobody thought to include. Stanford's HELM project argued in 2022 that models should be judged on several measures, such as accuracy, robustness, fairness and toxicity, not accuracy alone6.
Example
A UK bank building a customer-service chatbot might keep a private eval of 500 real customer questions with approved answers, and rerun it every time it changes the prompt or model.
Why it matters
Evals are how AI companies and regulators decide whether a model is ready to release. Weak evals are a common reason AI products behave worse in the real world than in demos.
Questions people ask
What is the difference between evals and benchmarks?
A benchmark is a public, standard test that everyone runs the same way, such as MMLU. Evals is the wider term: it also covers private tests a company builds for its own product, and safety tests such as red teaming4.
What is an LLM-as-a-judge eval?
One where another AI model grades the answers instead of a person. Research found strong judges such as GPT-4 matched human preferences over 80% of the time, but showed biases such as favouring longer answers3.
Related terms
Tools that use it
- Evidently AIOpen-source framework for testing and monitoring LLM apps, AI agents and machine learning models.
- Future AGIOpen-source platform for testing, evaluating and monitoring AI agents, with simulations, tracing and guardrails.
- OpenlayerPlatform for testing, monitoring and governing a company's AI models and agents
- LangWatchTesting, evaluation and monitoring platform for AI agents, with simulated users and red-teaming
Sources
- OpenAI (2023), GPT-4 Technical Report, arXiv:2303.08774
- OpenAI, openai/evals GitHub repository README
- Zheng et al. (2023), Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena, arXiv:2306.05685
- UK AI Security Institute and Meridian Labs, Inspect: an open-source framework for large language model evaluations
- METR (29 January 2026), Time Horizon 1.1
- Liang et al. (2022), Holistic Evaluation of Language Models, arXiv:2211.09110