FacebookExternalHit
Meta · Fetches shared links to build previews (title, description, image) on Facebook, Instagram, Messenger, and related Meta apps
By Vinespire Editorial Team, Editorial ·
Update: Verified against Meta operator documentation for FacebookExternalHit.
This page is the operator/profile guide for FacebookExternalHit. To paste a full robots.txt and check FacebookExternalHit plus other AI crawlers in one pass, use the Robots.txt AI Crawler Validator. Short definitional entries also live in the AI search glossary.
Related free utilities are in free AI SEO tools.
What is FacebookExternalHit?
FacebookExternalHit is Meta’s long-standing link-preview crawler. Official documentation states its primary purpose is to crawl content of an app or website shared on Meta’s family of apps so Meta can gather, cache, and display information such as title, description, and thumbnail image. Typical User-Agent strings include facebookexternalhit/1.1 with a reference URL, or facebookexternalhit/1.1 alone. Meta documents server requirements (for example gzip/deflate), Open Graph placement considerations, and tools such as the Sharing Debugger to force a recrawl. Critically, Meta notes that FacebookExternalHit might bypass robots.txt when performing security or integrity checks (such as malware checks), so respectsRobotsTxt is partial rather than a blanket yes. This agent is not Meta-ExternalAgent and should not be configured as if it were a training-only AI crawler; blocking it can break or degrade link previews when people share your URLs on Meta platforms. It remains adjacent to AI search work because many brands see FacebookExternalHit traffic alongside newer Meta AI agents and need accurate identification in logs and bot management rules.
How to block FacebookExternalHit
Add this group to robots.txt to disallow facebookexternalhit. Path rules can be narrowed if you only need to protect parts of the site.
User-agent: facebookexternalhit Disallow: /
How to allow FacebookExternalHit
For most public marketing and documentation sites, allowing well-behaved AI agents is the default recommended stance for AI visibility—while remaining a factual robots.txt Allow rule you can reverse later.
User-agent: facebookexternalhit Allow: /
Should you block FacebookExternalHit?
Most public marketing and content sites allow FacebookExternalHit so shared URLs render correct titles, descriptions, and images on Facebook, Instagram, and Messenger. Blocking can harm social distribution even when AI training policy is restrictive. If you must reduce traffic, prefer rate limits and CDN controls after verifying requests are legitimate Meta traffic, and understand Meta may still perform certain security-related fetches that bypass robots.txt. Do not use a FacebookExternalHit block as a substitute for Meta-ExternalAgent training policy—they are different agents. Keep Open Graph tags accurate for allowed fetches. Private or unlisted content should not rely on robots.txt; use authentication. Re-test previews with Meta’s Sharing Debugger after major robots or WAF changes.
Category: Dataset / other
Last verified: . Re-check operator docs after major crawler announcements.
Related bots (Dataset / other)
- Google-ExtendedGoogleControls use of content for Gemini and Google AI features (not classic Googlebot search)
- CCBotCommon CrawlBuilds the Common Crawl open web dataset used by many AI researchers and companies
- Applebot-ExtendedApplePreferences for Apple Intelligence / generative features (related to Applebot)
FacebookExternalHit FAQ
- Meta documents robots.txt preferences for its crawlers but notes FacebookExternalHit might bypass robots.txt for security or integrity checks. Treat compliance as partial for those cases.