Human in the loop

The High-Level Expert Group on AI, an independent expert group set up by the European Commission, describes three governance mechanisms for human oversight. Human-in-the-loop means a person can intervene in every decision cycle. Human-on-the-loop means a person can intervene during the design cycle and monitor the system while it runs. Human-in-command means a person oversees the system’s overall activity, including its economic, societal, legal and ethical impact, and can decide when and how to use the system, which can include overriding a decision¹. The guidelines say intervention in every cycle is in many cases neither possible nor desirable. They also say that, other things being equal, the less oversight a person can exercise over an AI system, the more extensive testing and stricter governance is required¹.

NIST’s AI Risk Management Framework does not use the name HITL. It says human roles and responsibilities in decision-making and oversight need to be clearly defined and differentiated, and that human-AI configurations can span from fully autonomous to fully manual². A system might decide on its own, defer a decision to a human expert, or be used by a human decision-maker as an additional opinion². Some systems may not require human oversight. NIST’s example is a model used to improve video compression. Other systems may specifically require it².

NIST says human-AI interaction results vary. In some perceptual-based judgement tasks, the AI part can amplify human biases, so the combined decision is more biased than the person or the AI alone². It says further study is still needed on how far people are empowered and incentivised to challenge an AI output, and that data on how often people overrule an output in deployed systems, and why, may be useful to collect and analyse². Anthropic describes agents pausing for human feedback at checkpoints or when they meet a blocker³. The OpenAI Agents SDK pauses a run until a person approves or rejects a sensitive tool call, then resumes from the saved state⁴.