Humanity's Last Exam

In short

Humanity's Last Exam (HLE) is an AI benchmark of 2,500 very hard questions written by subject experts across dozens of fields. It was built in 2025 to be tougher than older tests that top AI models had already mastered.

Also called: HLE, Humanity's Last Exam benchmark
Topic: Testing and benchmarks
Level: Beginner
First described: Created by the Center for AI Safety and Scale AI, led by Long Phan, Alice Gatti, Ziwen Han, Nathaniel Li and others, and released in January 2025¹.

HLE was released in January 2025 by the Center for AI Safety and Scale AI¹². Its authors built it because models were scoring over 90% on popular tests such as MMLU, which made those tests useless for comparing the best systems¹.

It has 2,500 questions across subjects including maths, the humanities and the natural sciences¹. Questions came from nearly 1,000 expert contributors at over 500 institutions in 50 countries³. Each has one checkable answer, and questions were kept only if they could not be quickly found online¹. About 14% need a diagram or image, and 24% are multiple choice³.

Early models did badly. Stanford's AI Index reported a top score of 8.80% in its 2025 edition⁴. Progress has been fast since. On Scale AI's leaderboard in October 2026, the top model scored 54.8%³.

The test has flaws. In July 2025 the research group FutureHouse estimated that about 29% of the chemistry and biology answers conflicted with published research⁵. The HLE team's own review found about 18% of a subset problematic, and now maintains a continuously corrected version, HLE-Rolling⁵³. The paper was published in Nature in January 2026².

HLE measures closed-ended academic questions. Its authors say a high score would not by itself show autonomous research ability³.

Example

A typical style of HLE question asks for one precise fact that is hard to look up. FutureHouse criticised one such question, about the rarest noble gas on Earth in 2002, as trivia with a disputed answer⁵.

Why it matters

It is one of the headline numbers AI companies now use to show their models are pushing past older tests. Its known errors are a reminder that even "hard" benchmarks need checking.

Questions people ask

Why is it called Humanity's Last Exam?

Its creators meant it to be the final closed-ended academic benchmark of its kind, with questions at the frontier of human expert knowledge¹.

What is the highest score on Humanity's Last Exam?

It changes quickly. On Scale AI's leaderboard on 11 October 2026, the top model scored 54.8%³. Scores vary between leaderboards because some allow web search or tools.

Are Humanity's Last Exam answers correct?

Not all of them. FutureHouse estimated about 29% of chemistry and biology answers were contradicted by published research, and the HLE team found about 18% of a subset problematic⁵.

Related terms

Sources

Back to the glossary