OpenAI, Google, Microsoft and Anthropic Back Shared AI Training Opt-Out

By Suad Seferi · 1 November 2026 at 13:19

OpenAI, Google, Microsoft and Anthropic are backing a common mechanism that would allow website owners to signal that their content should not be used to train generative artificial intelligence models. The companies announced their support on October 30, 2026, following the publication of a technical proposal developed through discussions at the Internet Engineering Task Force (IETF). The proposed system would allow publishers to express their AI training preferences through a single instruction in their website's robots.txt file, rather than maintaining separate restrictions for different AI companies. It could simplify a growing problem for publishers that want their content to remain publicly accessible without automatically making it available for AI model training. However, the mechanism is not yet an approved internet standard, and full implementation remains pending. One Preference Across Multiple AI Providers Website owners currently face a fragmented system for controlling how AI companies access and use their content. OpenAI provides controls for GPTBot, Google offers Google-Extended, and other providers maintain their own crawler-specific rules. This requires publishers to identify individual services, understand their policies and maintain different instructions as AI companies introduce new systems. The new proposal introduces a shared policy token called __usage_ai-training. A website operator wishing to express a preference against AI training across an entire website could add the following instructions to its robots.txt file: User-Agent: __usage_ai-training Disallow: / Participating AI providers would interpret the signal as a request not to use the website's content for generative AI training. The mechanism also allows website owners to specify different preferences for individual directories or sections. The technical proposal was submitted by Kevin Kelley of Anthropic on October 30. It uses existing robots.txt syntax rather than introducing a new file format. What OpenAI, Google and Microsoft Have Announced The companies have indicated different implementation schedules. OpenAI says it intends to support the shared signal in the coming months. Its existing crawler controls will remain available. Google plans to introduce support during 2027. Where a website uses the common training restriction without explicitly configuring Google-Extended, Google intends to interpret that restriction as a Google-Extended opt-out. Microsoft has announced February 1, 2027, as the date when it will begin recognising the new preferences as it crawls websites. Microsoft is also introducing separate controls for AI training and AI grounding, which refers to the use of web content to inform AI-generated answers in services such as Bing and Copilot. Its approach would allow publishers to restrict model training while continuing to make their content available for certain AI-generated responses. Anthropic has already begun implementing support for the proposed mechanism, according to Kelley, although deployment across its services remains in progress. Blocking AI Training Is Not the Same as Blocking AI Access The distinction between AI training and AI access is important. The proposed signal concerns the use of website content to train generative models. It does not automatically prevent AI crawlers from visiting a website, nor does it universally prevent AI assistants from using publicly available information when generating responses. It also does not retroactively remove information from models that have already been trained. Website owners therefore need to distinguish between three activities: allowing automated systems to crawl their pages, permitting content to be used for model training, and allowing content to inform AI-generated answers. Those activities are related, but they are not identical. Microsoft's new controls explicitly separate training from grounding. Google's proposed implementation introduces another consideration because Google-Extended settings can affect eligibility for content to appear in Gemini app responses. Publishers will still need to understand how individual providers interpret their preferences. Why the Proposal Matters for Independent Publishers The announcement comes amid continuing disputes over the use of online content by AI developers. News organisations, independent publishers and other website operators have raised concerns about their material being used to train commercial AI systems without sufficiently clear permission or compensation arrangements. A common training preference could reduce the technical work required to communicate those choices. For smaller publishers, this is particularly relevant. Maintaining separate rules for a growing number of AI crawlers can be difficult, especially when technical resources are limited. The new approach would provide a common method for expressing preferences, although its effectiveness would depend on how widely AI providers adopt and respect it. It would not establish copyright ownership, create licensing agreements or guarantee payment for content used by AI companies. Nor would it resolve the separate issue of AI-generated answers reducing traffic to original sources. A Proposal Awaiting Wider Adoption The shared mechanism remains an individual Internet-Draft rather than a formally approved IETF standard. Work continues within the organisation's AI Preferences working group, and the participating companies have not committed to identical implementation dates. Microsoft has already published a transition timetable for website operators using its existing NoArchive and NoCache preferences, including a March 1, 2027, deadline for maintaining certain restrictions. The immediate development is therefore an agreement on a common technical approach, not the introduction of a fully operational industry-wide restriction. The next test will come as the companies begin deploying their implementations in 2027…