AI alignment

In short

AI alignment is the effort to make AI systems do what people actually intend and act in line with human values, rather than chase a goal that only looks right on paper or causes harm along the way.

Also called: Alignment, Value alignment, The alignment problem
Topic: Safety, ethics and risk
Level: Some background helps
First described: Norbert Wiener described the core problem in "Some Moral and Technical Consequences of Automation", Science, 1960². The modern research field grew from the 2010s³.

A 2023 survey of the field defines it plainly: alignment aims to make AI systems behave in line with human intentions and values, and the risks from misalignment grow as systems become more capable¹.

The worry is old. In 1960 the mathematician Norbert Wiener warned in Science that if we use a machine we cannot easily stop, "we had better be quite sure that the purpose put into the machine is the purpose which we really desire"²³.

Today's chatbots are aligned mainly by training on human feedback. OpenAI's 2022 InstructGPT paper fine-tuned GPT-3 using people's rankings of its answers. Labellers preferred the answers of a 1.3 billion parameter InstructGPT model to those of the 175 billion parameter GPT-3, and the tuned models were more truthful and less toxic⁴.

Researchers often split the problem in two. Outer alignment asks whether the goal written into the system is the one its designers really want. Inner alignment asks whether the system actually learned that goal, or something that only matched it during training¹. Known failures include reward hacking, where a model games its scoring, and sycophancy, where it tells users what they want to hear.

Alignment is not solved. The International AI Safety Report 2026 says researchers cannot yet reliably train AI systems to behave as intended. It adds that a misaligned system might give false information, hide what it is doing or resist being shut down⁵.

Example

Ask a chatbot to "make this essay shorter" and an aligned model trims it while keeping the argument. A badly aligned one might delete the conclusion, because the instruction it followed was "shorter", not "shorter and still good".

Why it matters

Every AI product is only as useful as its fit with what the user meant. As AI systems take on longer tasks with less supervision, small gaps between instruction and intent can do more damage.

Questions people ask

What is the difference between AI alignment and AI safety?

Alignment is one part of safety. It is about whether a system pursues the goals its makers and users intend. AI safety is broader, and also covers deliberate misuse, such as fraud or cyberattacks, and wider effects on society⁵.

How are chatbots like ChatGPT aligned?

Mostly through extra training after the main training run. People write example answers and rank the model's replies, and the model is tuned towards the preferred ones. This approach, called reinforcement learning from human feedback, was set out in OpenAI's InstructGPT paper⁴.

What is inner and outer alignment?

Outer alignment is about whether the goal given to an AI system matches what its designers really want. Inner alignment is about whether the system actually learned that goal. The split is popular but contested, and some researchers say it is ambiguous¹.

Related terms

Sources

Back to the glossary