X-Robots-Tag
Definition
X-Robots-Tag is an HTTP response header that delivers robots directives such as noindex or nofollow without embedding a meta tag in HTML.
By Vinespire Editorial Team, Editorial ·
This term is part of the full AI search glossary.
Full definition
Headers work for HTML and non-HTML resources (PDFs, images) where a meta tag is awkward or impossible. Syntax mirrors meta robots values: noindex, none, noarchive, and similar directives supported by major engines.
When both meta robots and X-Robots-Tag appear, crawlers combine restrictive rules. Prefer one clear source of truth per URL to avoid conflicts during deployments.
CDN and edge configurations often inject X-Robots-Tag for whole path prefixes (for example /staging/). See Meta Robots, noindex, and CDN Caching.
Example
X-Robots-Tag: noindex, nofollow