On October 30, 2026, Google Search Central announced plans to support a shared AI training control in robots.txt. According to John Mueller, Google Search Advocate, this update helps website owners express their AI training preferences across multiple AI providers in a single step starting in 2027.
Key takeaways
- Google announced plans to support a shared AI training control in robots.txt.
- The update is based on work from the AIPREF working group at the Internet Engineering Task Force.
- Publisher choices will allow opting out of training across participating AI providers.
- Existing company-specific controls like Google-Extended will continue to function alongside the joint signal.
How will the shared AI training control work?
Google Search Central reports that website owners will use a shared policy token inside robots.txt. When a publisher uses this token to opt out of training and does not explicitly specify Google-Extended, Google plans to treat that instruction as a disallowance for Google-Extended. Other AI providers can handle the signal similarly.
Existing company-specific options will remain active alongside this joint signal. Publishers can use the industry signal for broad choices while tailoring instructions to individual developers like Google.
What are the rules for Google-Extended and Gemini Apps?
Websites that specifically allow Google-Extended remain eligible to appear in responses and receive clicks through grounding in Gemini Apps. This draft proposal responds to feedback from publishers, civil society groups, regulators, and other stakeholders.
The mechanism aligns with the EU Code of Practice for General Purpose AI Model Providers under the EU AI Act. This act recognizes robots.txt as a primary method for web publishers to express machine-readable preferences.
What happens next?
Google aims to support this new control alongside other AI providers in 2027 and will share progress updates. The company will continue working with stakeholders to refine and standardize the vocabulary through the Internet Engineering Task Force working group as multistakeholder discussions continue.
Frequently asked questions
Google Search Central announced plans to support a shared AI training control in robots.txt to help websites express their AI training preferences across multiple AI providers in a single step.
Google aims to support this new control along with other AI providers in 2027 and will provide updates on its progress.
Website owners will be able to use a shared policy token within robots.txt to set broad preferences across participating AI providers without managing separate instructions for every developer.
Existing company-specific controls like Google-Extended will continue to function alongside the new joint signal so website owners can tailor instructions to individual AI developers.
The AI training definition was developed at the Internet Engineering Task Force by the AIPREF working group.
Based on reporting by Google Search Central. Read the original article
