aHrefs Bot

Operator

Ahrefs

Type

Crawler

Official page

ahrefs.com

robots.txt token

AhrefsBot

Follows robots.txt

Yes, according to Ahrefs.

Runs JavaScript

Yes, according to Ahrefs.

Published IP ranges

api.ahrefs.com/v3/public/crawler-ip-ranges

In Vireo

Excluded, reported under SEO tools

What it does

Builds the database behind Ahrefs (an SEO and marketing tool) and Yep (Ahrefs' search engine).

Powers the database for both Ahrefs, a marketing intelligence platform, and Yep, an independent, privacy-focused search engine.

Ahrefs

User agent

Requests from aHrefs Bot carry user agents like these:

AhrefsBot.Feeds v0.1; http://ahrefs.com/

Mozilla/5.0 (compatible; AhrefsBot/3.1; +http://ahrefs.com/robot/

Mozilla/5.0 (compatible; AhrefsBot/5.0; +http://ahrefs.com/robot/)

Device Detector identifies it with this pattern (a case-insensitive regular expression):

AhrefsBot

Anyone can put any name in a user agent. Where the operator publishes IP ranges, checking the request's address against them is the way to confirm a visit is genuine.

How to block or allow aHrefs Bot

Add a group for its token to the robots.txt file at the root of your site. To keep it out of the whole site:

User-agent: AhrefsBot
Disallow: /

To let it crawl everything, use an empty Disallow: or Allow: / instead. WordPress serves a virtual robots.txt that many SEO plugins let you edit. A real robots.txt file uploaded to the site root takes its place.

User-agent token in robots.txt: AhrefsBot

Ahrefs

Obeys robots.txt: Yes

Ahrefs

robots.txt is a request, so it only works on crawlers that honour it. A bot that ignores it has to be blocked at the server or firewall, by user agent or IP address.

Worth knowing

Ahrefs renders JavaScript, and while rendering a page it can fetch several assets at once, faster than your crawl-delay. Its separate AhrefsSiteAudit bot can be set by a verified site owner to ignore robots.txt on their own site.

When our bots render a page, they may request multiple assets (e.g., images, scripts, stylesheets) simultaneously, which can result in more frequent requests appearing in server logs than allowed by the crawl-delay setting.

Ahrefs

Does aHrefs Bot run JavaScript?

Yes. Ahrefs says it does.

Crawl-delay is strictly followed when requesting HTML pages, ensuring that we do not exceed the specified rate limits. However, it cannot be followed when rendering JavaScript.

Ahrefs

How Vireo counts aHrefs Bot

Vireo keeps requests from aHrefs Bot out of your pageviews and visitors and counts them on its bot report under SEO tools, so you can see how much it left out rather than wonder why your server logs show more traffic.

Checked by running Vireo Analytics' own bot filter over aHrefs Bot's user agents.

Sources

• Ahrefs: crawler documentation (checked October 2026)
• Matomo Device Detector, bots.yml (LGPL-3.0)