Bot traffic

Where bot traffic comes from

Most of the requests a public website gets are automated. Search engines crawl to keep their index fresh. SEO tools crawl to map links. Uptime monitors request a page every minute or five. Social networks fetch a page to build a preview card when someone shares the link. AI companies crawl for training data and to answer questions. Scripts and scrapers fill in the rest.

None of these are readers, and a report that counts them as visits overstates your audience. Bots that run JavaScript can also slip into tag-based tools like Google Analytics, which only filters a list of known bots.

How Vireo recognises a bot

Vireo checks every request when it asks WordPress for a page, before any JavaScript runs. That way it also sees crawlers that never execute scripts, which most crawlers don't.

It matches over 800 known bots by user agent, using Matomo Device Detector's list plus one Vireo keeps itself. It also catches requests with no user agent, automated browsers that set the flags testing tools put on themselves even when they claim to be Chrome, and pages a browser prefetched or prerendered without anyone looking. On Cloudflare, it leaves out requests Cloudflare marks as a verified bot, challenges or scores as a likely threat.

One visitor counts for at most 5 views of a page and 50 views in total a day. Views past that cap go to the bot report too, so a script reloading one URL can't inflate your numbers.

What you see

Under the totals on the Overview, a line says how many requests Vireo kept out and what share of everything it saw, split by kind: AI crawlers, search engines, SEO tools, uptime monitors, link previews, headless browsers, scripts and clients, archivers and validators, prefetch and prerender, repeated views past the daily cap, and other bots.

To check one user agent, go to Settings, Tracking, Bot filter test, or paste it into the bot checker on this site.

Pitfalls

On a site with full-page caching, a cached page is answered before WordPress runs. Vireo can't see those requests, so the real bot count is higher than the report shows. Your human visitors are still counted, because the tracker runs in their browser.

A user agent is only a claim. A scraper can call itself Googlebot. Where it matters, check the request's IP address against the ranges the operator publishes, which the bot directory links to where they exist.