Facebook Crawler

Facebook Crawler is a link preview fetcher for a social network run by Meta.

Operator
Meta
Type
Social Media Agent
Official page
developers.facebook.com
Follows robots.txt
Partly, according to Meta.
In Vireo
Excluded, reported under Link previews

What it does

Fetches pages shared on Facebook, Instagram or Messenger to build link previews (title, description, thumbnail).

The primary purpose of FacebookExternalHit is to crawl the content of an app or website that was shared on one of Meta’s family of apps, such as Facebook, Instagram, or Messenger.

Meta

User agent

Requests from Facebook Crawler carry user agents like these:

facebookexternalua
facebookcatalog/1.0
facebookplatform/1.0 (+http://developers.facebook.com)

Device Detector identifies it with this pattern (a case-insensitive regular expression):

facebook(?:catalog|externalhit|externalua|platform|scraper)

Anyone can put any name in a user agent. Where the operator publishes IP ranges, checking the request's address against them is the way to confirm a visit is genuine.

Worth knowing

Open Graph tags have to sit in the first 1 MB of the page or they get cut off.

Any Open Graph properties need to be listed before the first 1 MB of your website or app, or it will be cutoff.

Meta

How Vireo counts Facebook Crawler

Vireo keeps requests from Facebook Crawler out of your pageviews and visitors and counts them on its bot report under Link previews, so you can see how much it left out rather than wonder why your server logs show more traffic.

Checked by running Vireo Analytics' own bot filter over Facebook Crawler's user agents.

Sources

Other bots from Meta

More link previews

All link previews

See the bots on your own site

Vireo Analytics is a free WordPress plugin. It keeps over 800 known crawlers out of your numbers and shows you what it filtered, by category.

Get Vireo free on WordPress.org