AhrefsSiteAudit

AhrefsSiteAudit is a site monitor run by Ahrefs.

Operator
Ahrefs
Type
Site Monitor
Official page
ahrefs.com
robots.txt token
AhrefsSiteAudit
Sends readers back
No, it serves an SEO tool
Follows robots.txt
Partly, according to Ahrefs.
Runs JavaScript
Yes, according to Ahrefs.
Published IP ranges
api.ahrefs.com/v3/public/crawler-ip-ranges
In Vireo
Excluded, reported under Uptime monitors

What it does

Crawls sites that Ahrefs users add to the Site Audit tool to find technical and on-page SEO problems.

Powers Ahrefs’ Site Audit tool. Ahrefs users can use Site Audit to analyze websites and find both technical SEO and on-page SEO issues.

Ahrefs

User agent

Requests from AhrefsSiteAudit carry a user agent like this:

AhrefsSiteAudit/6.1; http://ahrefs.com/robot/site-audit

Device Detector identifies it with this pattern (a case-insensitive regular expression):

AhrefsSiteAudit

Anyone can put any name in a user agent. Where the operator publishes IP ranges, checking the request's address against them is the way to confirm a visit is genuine.

How to block or allow AhrefsSiteAudit

Add a group for its token to the robots.txt file at the root of your site. To keep it out of the whole site:

User-agent: AhrefsSiteAudit
Disallow: /

To let it crawl everything, use an empty Disallow: or Allow: / instead. WordPress serves a virtual robots.txt that many SEO plugins let you edit. A real robots.txt file uploaded to the site root takes its place.

User-agent token in robots.txt:

Ahrefs

Only verified site owners can allow AhrefsSiteAudit crawler to disobey robots.txt on their site so they can check for issues on the site sections normally disallowed for crawling.

Ahrefs

robots.txt is a request, so it only works on crawlers that honour it. A bot that ignores it has to be blocked at the server or firewall, by user agent or IP address.

Worth knowing

It caps itself at 30 URLs a minute unless a verified site owner raises the speed.

AhrefsSiteAudit bot prevents excessive load on website servers by limiting crawling to 30 URLs per min max.

Ahrefs

Does AhrefsSiteAudit run JavaScript?

Yes. Ahrefs says it does.

By crawling and rendering pages, we help identify improvements that can boost visibility, loading speed, and overall user experience.

Ahrefs

Does AhrefsSiteAudit send readers back?

No. AhrefsSiteAudit gathers data for Ahrefs's SEO tools, for the people who use them. It isn't a source of readers.

How Vireo counts AhrefsSiteAudit

Vireo keeps requests from AhrefsSiteAudit out of your pageviews and visitors and counts them on its bot report under Uptime monitors, so you can see how much it left out rather than wonder why your server logs show more traffic.

Checked by running Vireo Analytics' own bot filter over AhrefsSiteAudit's user agents.

Sources

Other bots from Ahrefs

More uptime monitors

All uptime monitors

See the bots on your own site

Vireo Analytics is a free WordPress plugin. It keeps over 800 known crawlers out of your numbers and shows you what it filtered, by category.

Get Vireo free on WordPress.org