AhrefsSiteAudit
AhrefsSiteAudit is a site monitor run by Ahrefs.
- Operator
- Ahrefs
- Type
- Site Monitor
- Official page
- ahrefs.com
- robots.txt token
AhrefsSiteAudit- Sends readers back
- No, it serves an SEO tool
- Follows robots.txt
- Partly, according to Ahrefs.
- Runs JavaScript
- Yes, according to Ahrefs.
- Published IP ranges
- api.ahrefs.com/v3/public/crawler-ip-ranges
- In Vireo
- Excluded, reported under Uptime monitors
What it does
Crawls sites that Ahrefs users add to the Site Audit tool to find technical and on-page SEO problems.
Powers Ahrefs’ Site Audit tool. Ahrefs users can use Site Audit to analyze websites and find both technical SEO and on-page SEO issues.
Ahrefs
User agent
Requests from AhrefsSiteAudit carry a user agent like this:
AhrefsSiteAudit/6.1; http://ahrefs.com/robot/site-auditDevice Detector identifies it with this pattern (a case-insensitive regular expression):
AhrefsSiteAuditAnyone can put any name in a user agent. Where the operator publishes IP ranges, checking the request's address against them is the way to confirm a visit is genuine.
How to block or allow AhrefsSiteAudit
Add a group for its token to the robots.txt file at the root of your site. To keep it out of the whole site:
User-agent: AhrefsSiteAudit
Disallow: /To let it crawl everything, use an empty Disallow: or Allow: / instead. WordPress serves a virtual robots.txt that many SEO plugins let you edit. A real robots.txt file uploaded to the site root takes its place.
User-agent token in robots.txt:
Ahrefs
Only verified site owners can allow AhrefsSiteAudit crawler to disobey robots.txt on their site so they can check for issues on the site sections normally disallowed for crawling.
Ahrefs
robots.txt is a request, so it only works on crawlers that honour it. A bot that ignores it has to be blocked at the server or firewall, by user agent or IP address.
Worth knowing
It caps itself at 30 URLs a minute unless a verified site owner raises the speed.
AhrefsSiteAudit bot prevents excessive load on website servers by limiting crawling to 30 URLs per min max.
Ahrefs
Does AhrefsSiteAudit run JavaScript?
Yes. Ahrefs says it does.
By crawling and rendering pages, we help identify improvements that can boost visibility, loading speed, and overall user experience.
Ahrefs
Does AhrefsSiteAudit send readers back?
No. AhrefsSiteAudit gathers data for Ahrefs's SEO tools, for the people who use them. It isn't a source of readers.
How Vireo counts AhrefsSiteAudit
Vireo keeps requests from AhrefsSiteAudit out of your pageviews and visitors and counts them on its bot report under Uptime monitors, so you can see how much it left out rather than wonder why your server logs show more traffic.
Checked by running Vireo Analytics' own bot filter over AhrefsSiteAudit's user agents.
Sources
Other bots from Ahrefs
More uptime monitors
- 360 Monitoring (Plesk International GmbH)
- 360JK (360 Security Technology Inc.)
- Alexa Site Audit (Alexa Internet)
- Amazon ELB (AWS)
- AppSignalBot (AppSignal)
- BBC Forge URL Monitor (BBC)
- BBC Page Monitor (BBC)
- Better Uptime Bot (Better Stack)
- Blackbox Exporter (Prometheus)
- Buck (Hypefactors A/S)
See the bots on your own site
Vireo Analytics is a free WordPress plugin. It keeps over 800 known crawlers out of your numbers and shows you what it filtered, by category.
Get Vireo free on WordPress.org