Fairness in AI
In short
Fairness in AI means designing and checking AI systems so they do not treat people unjustly because of traits such as race, sex or disability. It is hard because there are several mathematical definitions of "fair", and they can conflict.
- Also called
- AI fairness, Algorithmic fairness, Fair machine learning
- Topic
- Safety, ethics and risk
- Level
- Some background helps
- First described
- Dwork, Hardt, Pitassi, Reingold and Zemel formalised individual fairness in "Fairness Through Awareness", posted in April 20112.
The US standards body NIST says fairness in AI covers equality and equity, including tackling harmful bias and discrimination. It warns that fairness is hard to define, because ideas of what is fair differ between cultures and between uses1.
Computer scientists have tried to pin it down. A 2011 paper by Cynthia Dwork and colleagues proposed that similar individuals should be treated similarly. It also discussed statistical parity, where the people receiving a decision mirror the make-up of the wider population2.
These definitions can clash. In 2016 Jon Kleinberg, Sendhil Mullainathan and Manish Raghavan proved that three common fairness conditions for risk scores cannot all be met at once, except in narrow special cases3. So building a fair system means choosing which kind of fairness matters most for the job, and saying so.
Fairness is also wider than balanced numbers. NIST notes that a system whose predictions are balanced across groups may still be inaccessible to disabled people or widen existing inequalities1.
In the UK, fairness is a legal principle in data protection. The ICO's guidance on AI says fairness under data protection law is not only about discrimination, and that organisations must also meet their duties under the Equality Act 20104.
Example
In 2016 ProPublica studied COMPAS, a tool used in US courts to predict reoffending. Looking at over 7,000 people in Broward County, Florida, it found Black defendants were almost twice as likely as white defendants to be labelled higher risk without going on to reoffend5. The case is now a standard teaching example of fairness definitions in conflict3.
Why it matters
AI is used in decisions about credit, jobs, benefits and policing. Which definition of fairness a system uses decides who bears the cost of its mistakes.
Questions people ask
Can an AI system be completely fair?
Not by every measure at once. Kleinberg and colleagues proved in 2016 that three common fairness conditions for risk scores cannot all hold together, except in special cases3. Builders have to choose and justify which kind of fairness to prioritise.
What is the difference between fairness and bias in AI?
Bias describes a systematic skew in how a system treats groups. Fairness is the goal: treating people justly. NIST says removing harmful bias does not by itself make a system fair1.
Is AI fairness a legal requirement in the UK?
Yes, in two ways. Data protection law requires personal data to be processed fairly, and the ICO has published guidance on what that means for AI. Organisations must also comply with the Equality Act 20104.
Related terms
Sources
- NIST AI 100-1 (January 2023), AI Risk Management Framework 1.0, section 3.7
- Dwork et al. (2011), Fairness Through Awareness, arXiv:1104.3913
- Kleinberg, Mullainathan and Raghavan (2016), Inherent Trade-Offs in the Fair Determination of Risk Scores, arXiv:1609.05807
- ICO, Guidance on AI and data protection: How do we ensure fairness in AI?
- Angwin, Larson, Mattu and Kirchner (ProPublica, 23 May 2016), Machine Bias