Rogerbot

Rogerbot is a crawler run by Moz.

Operator
Moz
Type
Crawler
Official page
moz.com
robots.txt token
rogerbot
Follows robots.txt
Yes, according to Moz.
In Vireo
Excluded, reported under SEO tools

What it does

Moz's site audit crawler for Moz Pro Campaigns and On-Demand Crawl.

Rogerbot is the Moz crawler for Moz Pro Campaign site audits.

Moz

User agent

Requests from Rogerbot carry user agents like these:

rogerbot/1.0 (http://www.moz.com/dp/rogerbot, rogerbot-crawler@moz.com)
Mozilla/5.0 (compatible; rogerBot/1.0; UrlCrawler; http://www.seomoz.org/dp/rogerbot)
rogerbot/1.0 (http://moz.com/help/pro/what-is-rogerbot-, rogerbot-crawler+shiny@moz.com)

Device Detector identifies it with this pattern (a case-insensitive regular expression):

rogerbot

Anyone can put any name in a user agent. Where the operator publishes IP ranges, checking the request's address against them is the way to confirm a visit is genuine.

How to block or allow Rogerbot

Add a group for its token to the robots.txt file at the root of your site. To keep it out of the whole site:

User-agent: rogerbot
Disallow: /

To let it crawl everything, use an empty Disallow: or Allow: / instead. WordPress serves a virtual robots.txt that many SEO plugins let you edit. A real robots.txt file uploaded to the site root takes its place.

The best option available is to identify our crawler by User-agent: rogerbot.

Moz

Rogerbot is built to obey robots.txt files.

Moz

robots.txt is a request, so it only works on crawlers that honour it. A bot that ignores it has to be blocked at the server or firewall, by user agent or IP address.

Worth knowing

No published IP range.

No, we do not have an IP range for Rogerbot as we do not use a static IP address or range of IP addresses.

Moz

How Vireo counts Rogerbot

Vireo keeps requests from Rogerbot out of your pageviews and visitors and counts them on its bot report under SEO tools, so you can see how much it left out rather than wonder why your server logs show more traffic.

Checked by running Vireo Analytics' own bot filter over Rogerbot's user agents.

Sources

Other bots from Moz

More seo tools

All seo tools

See the bots on your own site

Vireo Analytics is a free WordPress plugin. It keeps over 800 known crawlers out of your numbers and shows you what it filtered, by category.

Get Vireo free on WordPress.org