Opt-out (AI training)
In short
An AI training opt-out is a signal that specified content should not be used to train a model. Some are robots.txt rules published by AI companies. In the EU, rightholders can also reserve text-and-data-mining rights in a machine-readable way.
- Also called
- AI training opt-out, Do not train on my content
- Topic
- Safety, ethics and risk
- Level
- Beginner
- First described
- Article 4(3) of Directive (EU) 2019/790, of 17 April 2019, made the EU text-and-data-mining exception conditional on rightholders not having expressly reserved the use, including by machine-readable means for content made public online4.
OpenAI tells site owners to use separate robots.txt tags. Disallowing GPTBot indicates that the site's content should not be used to train OpenAI's generative models. OAI-SearchBot is for showing sites in ChatGPT search. The settings are independent, so a site can allow search and disallow training. If both are allowed, OpenAI may use one crawl for both. Visits by ChatGPT-User are different: OpenAI says robots.txt may not apply, because a person triggered the visit.1
Anthropic uses ClaudeBot to collect content that could contribute to training. Restricting ClaudeBot signals that the site's future materials should be excluded from training datasets. Claude-SearchBot is for search indexing, and Claude-User is for retrieval when a person asks a question. Anthropic says the bots honour robots.txt. Blocking an address instead may fail, because the crawler then cannot read the file. The rule has to be added for each host name the owner wants to cover.2
Google-Extended is not a separate crawler. Google says crawling still uses its existing user agents, and that the token lets a publisher manage whether crawled content may be used to train future Gemini models and for grounding. It does not affect inclusion in Google Search, and it is not a ranking signal.3
Company tags are not the only form. Article 4 of Directive (EU) 2019/790 tells member states to allow reproductions and extractions of lawfully accessible works for text and data mining. That exception applies only if the rightholder has not expressly reserved the use in an appropriate manner, such as machine-readable means when the content is public online. It does not affect the separate research exception in Article 3.4 Article 53 of the EU AI Act requires a general-purpose model provider to put in place a policy to comply with Union copyright law, including to identify and comply with an Article 4(3) reservation. Recital 105 says that, where the reservation has been made, the provider needs the rightholder's authorisation before mining those works. The explorer marks Article 53 as applying from 2 August 2025.56
Example
A news site disallows GPTBot and allows OAI-SearchBot. OpenAI says those settings are independent: a GPTBot disallow indicates the content should not be used to train its generative models, while allowing OAI-SearchBot is what lets a site be shown in ChatGPT search answers.1
Why it matters
Each control names a bot, a future training use, or a legal reservation. A training tag, a search tag and an EU text-and-data-mining reservation do not do the same job.134
Questions people ask
Does an AI training opt-out delete my data?
Not on the terms these providers publish. Anthropic says a ClaudeBot restriction is a signal about future materials and training datasets. Google-Extended concerns training of future Gemini models, and grounding, not removal from Google Search. OpenAI says a GPTBot disallow indicates the content should not be used to train its generative models.123
Does blocking a training bot block AI search?
Not necessarily. OpenAI's training and search tags are independent. Google says Google-Extended does not affect Google Search. Anthropic uses a separate Claude-SearchBot for search indexing. Blocking one of these leaves the others to their own rules.123
Is an AI training opt-out a legal right?
It depends where you are. OpenAI, Anthropic and Google publish robots.txt controls for their own systems. In the EU, Article 4 of the 2019 copyright directive withholds the text-and-data-mining exception where a rightholder has reserved the use. The EU AI Act requires general-purpose model providers to have a policy to identify and comply with that reservation.45
How do I opt a website out of AI training?
For these companies, add the robots.txt token they name: GPTBot, ClaudeBot, or Google-Extended. Anthropic says the rule belongs in robots.txt for every host name you want to cover, and that blocking an address may not stick. An EU reservation must be an express reservation in an appropriate manner. For public online content, the directive gives machine-readable means as the example and does not name one file format.1234
Related terms
Sources
- OpenAI, Overview of OpenAI Crawlers
- Anthropic, Does Anthropic crawl data from the web, and how can site owners block the crawler?
- Google, Google's common crawlers
- Directive (EU) 2019/790, Article 4, exception or limitation for text and data mining
- EU AI Act, Article 53, obligations for providers of general-purpose AI models
- EU AI Act, Recital 105