Does Google-Extended respect robots.txt?

Google documents product tokens such as Google-Extended for robots.txt preferences related to generative AI features. Follow Google’s crawler overview for current tokens and examples.

Is Google-Extended the same as Googlebot?

No. Googlebot is the primary crawler for Google Search. Google-Extended expresses preferences for Gemini / Google AI feature uses. Blocking Google-Extended is not the same as blocking Search.

Does blocking Google-Extended affect Google rankings?

Google has positioned Google-Extended as separate from classic Search crawling. Still verify Google’s current documentation for your use case, and do not block Googlebot unless you intend to affect Search.

Should I allow Google-Extended?

Allow if you want content available to documented Google AI features; block if you prefer opt-out for those uses while keeping Search. Align legal and SEO stakeholders.

Where do I put Google-Extended rules?

In robots.txt as User-agent: Google-Extended with Allow or Disallow paths, separate from your Googlebot group.

How do I test Google-Extended configuration?

Use Vinespire’s Robots.txt AI Validator for a quick registry check, then confirm against Google’s documentation and production robots.txt after deploy.

Google-Extended

Google · Controls use of content for Gemini and Google AI features (not classic Googlebot search)

What is Google-Extended?

Google-Extended is a product token used in robots.txt to express preferences about whether Google may use your site’s content for Gemini and certain Google AI / generative features. It is not a substitute for Googlebot: classic Search crawling and ranking generally continue to use Googlebot (and related search crawlers). Site owners who want to remain in Google Search while limiting AI training/grounding use cases often allow Googlebot and configure Google-Extended separately. Always read Google’s current crawler and robots.txt documentation for the authoritative list of user-agents and product tokens.

How to block Google-Extended

Add this group to robots.txt to disallow Google-Extended. Path rules can be narrowed if you only need to protect parts of the site.

User-agent: Google-Extended
Disallow: /

How to allow Google-Extended

For most public marketing and documentation sites, allowing well-behaved AI agents is the default recommended stance for AI visibility—while remaining a factual robots.txt Allow rule you can reverse later.

User-agent: Google-Extended
Allow: /

Should you block Google-Extended?

Blocking Google-Extended is a product preference about Google AI features, not a general “block Google” switch. If organic Search rankings are your priority, do not confuse Google-Extended with Googlebot. Many brands allow Google-Extended for AI feature inclusion; others block it for training/licensing reasons while still allowing Googlebot. Decide explicitly, document the choice, and re-check Google’s crawler overview when tokens change.

Category: Dataset / other

Official documentation (Google)

Last verified: 2026-07-18. Re-check operator docs after major crawler announcements.

Google-Extended FAQ

Google documents product tokens such as Google-Extended for robots.txt preferences related to generative AI features. Follow Google’s crawler overview for current tokens and examples.