Other known bots
Every other known bot: feed readers, security scanners, research crawlers, data companies and more. Vireo also catches unnamed ones by the words bots tend to put in their user agents, such as bot, crawler and spider, and reports those here too.
641 known bots from 370 operators. 48 have their own page with user agents, robots.txt tokens and sources.
| Bot | Operator | Type |
|---|---|---|
| 1001FirmsBot | Not known | Crawler |
| 2GDPR | 2GDPR | Service Agent |
| 2ip | Not known | Crawler |
| ABEvalBot | Not known | Crawler |
| Abonti | Not known | Crawler |
| AccessibleWebBot | Accessible Web Inc. | Crawler |
| AccompanyBot | Accompani, Inc | Crawler |
| AdAuth | Not known | Crawler |
| Adbeat | PPC Labs LLC | Crawler |
| AddSearchBot | AddSearch Oy | Crawler |
| ADMantX | Not known | |
| ADmantX Service Fetcher | Not known | Service bot |
| Adsbot | Not known | Crawler |
| Adscanner | AdScanner d.o.o | Crawler |
| AdsTxtCrawler | IAB Technology Laboratory, Inc. | Crawler |
| AdsTxtCrawlerTP | Not known | Crawler |
| adstxtlab.com | Jaohawi AB | Crawler |
| Aegis | Not known | Crawler |
| AffiliateSpyBot | Wayfinity Ltd | Crawler |
| AffsignalCrawler | Not known | Crawler |
| AgentDataBot | Not known | Listed by ai.robots.txt |
| aiHitBot | aiHit | Crawler |
| AliyunSecBot | Not known | Security Checker |
| Allloadin Favicon Bot | Not known | Crawler |
| AlltheWeb | Not known | Crawler |
| AlphaXCrawl | University of Passau | Crawler |
| Amazon AdBot | Amazon.com, Inc. | Crawler |
| Amazon Kendra | Amazon.com, Inc. | Crawler |
| Amazon Route53 Health Check | Amazon Web Services | Service Agent |
| Amorank Spider | Amorank | Crawler |
| Analytics SEO Crawler | Analytics SEO | Crawler |
| Ant | Ant.com Ltd. | Crawler |
| Apache | The Apache Software Foundation | Service Agent |
| ApifyBot | Not known | Listed by ai.robots.txt |
| ApifyWebsiteContentCrawler | Not known | Listed by ai.robots.txt |
| Arachni | Sarosys LLC | Security Checker |
| Aranet-SearchBot | Not known | Listed by ai.robots.txt |
| ArchiveBot | ArchiveTeam | Crawler |
| ArchiveBox | Not known | Crawler |
| Arquivo.pt | FCT|FCCN | Crawler |
| ARSNova Filter System | Particify Gerhardt & Weingarten OHG | Crawler |
| ArtelLeadRadar | Not known | Crawler |
| Asana | Asana, Inc. | Crawler |
| AspiegelBot | Huawei | Crawler |
| Assetnote | Assetnote Pty Ltd | Security Checker |
| atlassian-bot | Atlassian | Listed by ai.robots.txt |
| Automattic Analytics | Wordpress.org | Crawler |
| AzureAI-SearchBot | Not known | Listed by ai.robots.txt |
| Backlink-Check.de | Mediagreen Medienservice | Crawler |
| BacklinkCrawler | 2.0Promotion GbR | Crawler |
| BacklinksExtendedBot | Semrush Inc. | Crawler |
| BackupLand | ООО «КВАРТА» | Crawler |
| Barracuda Sentinel | Barracuda Networks, Inc. | Service Agent |
| BazQux Reader | Not known | Feed Fetcher |
| BDCbot | BIG Data Solucoes Em Tecnologia de Informatica LTDA | Crawler |
| BDFetch | Not known | Crawler |
| bedrockbot | Amazon | Listed by ai.robots.txt |
| Birdcrawlerbot | Swoppen Systems GmbH | Crawler |
| BitlyBot | Bitly, Inc. | Crawler |
| BitSight | BitSight Technologies, Inc. | Security Checker |
| BixelBot | Not known | Listed by ai.robots.txt |
| Blackboard Ally | Anthology Inc. | Crawler |
| Bloglines | Reply!, Inc. | Crawler |
| Bloglovin | Not known | Feed Fetcher |
| Blogtrottr | Blogtrottr Ltd | Feed Fetcher |
| BoardReader Blog Indexer | BoardReader | Crawler |
| Botify | BOTIFY SAS | Crawler |
| BrandVerity | BrandVerity, Inc. | Crawler |
| BrightBot | Bright Interactive Ltd | Crawler |
| Brightbot 1.0 | https://brightdata.com/brightbot | Listed by ai.robots.txt |
| BrightEdge | BrightEdge Technologies, Inc | Crawler |
| Browsershots | Browsershots.org | Service Agent |
| BUbiNG | The Laboratory for Web Algorithmics (LAW) | Crawler |
| BuddyBot | BuddyBotLearning | Listed by ai.robots.txt |
| BuiltWith | BuiltWith Pty Ltd | Crawler |
| CareerBot | career-x GmbH | Crawler |
| Castopod | Not known | Crawler |
| Castro 2 | Supertop | Service Agent |
| CensysInspect | Censys, Inc. | Security Checker |
| Channel3Bot | Not known | Listed by ai.robots.txt |
| Charlotte | Searchme, Inc. | Crawler |
| Chatwork LinkPreview | kubell Co., Ltd. | Service Agent |
| CheckerProxy | Not known | Security Checker |
| CheckMark Network | Exipert, Inc. | Crawler |
| Choosito | Choosito! Inc. | Crawler |
| Chrome Privacy Preserving Prefetch Proxy | Google Inc. | Service Agent |
| Cincraw | Not known | Crawler |
| CISPA Web Analyzer | CISPA - Helmholtz-Zentrum für Informationssicherheit gGmbH | Crawler |
| CLASSLA-web | Jožef Stefan Institute | Crawler |
| Clickagy | Clickagy, LLC | Crawler |
| Cliqzbot | 10betterpages GmbH | Crawler |
| CloudFlare AMP Fetcher | CloudFlare | Crawler |
| Cloudflare Custom Hostname Verification | CloudFlare | Service Agent |
| Cloudflare Health Checks | CloudFlare | Service Agent |
| CloudflareBrowserRenderingCrawler | Cloudflare that returns rendered website content for research, monitoring, and AI data workflows | Listed by ai.robots.txt |
| CloudServerMarketSpider | Not known | Crawler |
| CloudVertexBot | Not known | Listed by ai.robots.txt |
| CMS Experiment | Not known | Security Checker |
| CMSLookup | Not known | Security Checker |
| Cocolyzebot | VSI INNOVATION SAS | Crawler |
| colly | Not known | Crawler |
| CommaFeed | Not known | Feed Fetcher |
| COMODO DCV | Comodo Security Solutions, Inc. | Service Agent |
| Comscore | Not known | Crawler |
| Convertify | Convertify | Service Agent |
| Cookiebot | Cybot A/S | Crawler |
| CopyvioDetector | Ben Kurtovic | Crawler |
| Cortex Xpanse | Palo Alto Networks, Inc. | Security Checker |
| Cotoyogi | Joint Support-Center for Data Science Research (ROIS-DS) | Crawler |
| CragCrawler | CragSoftware, a Brazil-based software company specializing in data engineering and AI web scraping services | Listed by ai.robots.txt |
| Crawl4AI | Not known | Listed by ai.robots.txt |
| Crawldad | Not known | Crawler |
| crawlora-research | Not known | Security Checker |
| Crawlson | Crawlson | Crawler |
| Crawly Project | Not known | Crawler |
| CriteoBot | Not known | Crawler |
| CrowdTangle | CrowdTangle, Inc. | Crawler |
| CrystalSemanticsBot | Crystal Semantics Ltd. | Crawler |
| CSE-Scanner | Not known | Security Checker |
| CSS Certificate Spider | Certified Security Solutions | Crawler |
| CybaaAgent | Cybaa Ltd | Service Agent |
| CybaaBot | Cybaa Ltd | Security Checker |
| CyberFind Crawler | Find.tf | Crawler |
| Cyberscan | DGC Verwaltungs GmbH | Security Checker |
| Cypex | Cypex | Security Checker |
| DaspeedBot | DAWAP SARL | Crawler |
| datagnionbot | DATAGNION GMBH | Crawler |
| Datanyze | Datanyze | Crawler |
| Dataprovider | Dataprovider B.V. | Crawler |
| DataXu | Roku, Inc. | Service Agent |
| Datenbank Crawler | Datenbank | Listed by ai.robots.txt |
| Deep SEARCH 9 | Copyright Clearance Center, Inc. | Crawler |
| Deepfield Genome | Nokia Corporation | Crawler |
| deepnoc | deepnoc, GmbH | Crawler |
| DepSpid | Not known | Crawler |
| Detectify | Detectify AB | Security Checker |
| Disqus | Disqus, Inc. | Crawler |
| DnesAkceBot | Easy Call 09, s.r.o. | Crawler |
| DNSResearchBot | Not known | Crawler |
| Domain Codex | Erie Data Systems, LLC | Crawler |
| Domain Re-Animator Bot | Domain Re-Animator, LLC | Crawler |
| Domain Research Project | Trent Wiles | Crawler |
| DomainAppender | Profound Networks, LLC | Crawler |
| DomainCrawler | Not known | Crawler |
| DomainDrift | DRM3 Labs Corp. | Security Checker |
| DomainProbe | Not known | Security Checker |
| Domains Project | Not known | Crawler |
| DomainStatsBot | Domainstats Ltd | Crawler |
| DomCop Bot | Axeman Technology Solutions LLP | Crawler |
| Dormouse | Not known | Crawler |
| DoubaoBot | ByteDance | Listed by ai.robots.txt |
| Dubbotbot | DubBot | Crawler |
| ducks.party | Not known | Security Checker |
| DuplexWeb-Google | Google Inc. | Crawler |
| DVbot | DoubleVerify Inc. | Service Agent |
| eCairn-Grabber | eCairn | Crawler |
| Echobot Bot | Echobox | Listed by ai.robots.txt |
| EchoboxBot | Echobox | Listed by ai.robots.txt |
| EFF Do Not Track Policy Checker | Electronic Frontier Foundation | Crawler |
| EFF Do Not Track Verifier | Electronic Frontier Foundation | Crawler |
| EMail Exractor | Not known | Crawler |
| EmailWolf | Not known | |
| Embedly | A Medium, Corp. | Crawler |
| Entfer | Entfer Ltd. | Crawler |
| ERNIEBot | Baidu | Listed by ai.robots.txt |
| evc-batch | eVenture Capital Partners II, LLC | Crawler |
| Everyfeed | Not known | Feed Fetcher |
| Example3 | Not known | Crawler |
| ExaSearchBot | Exa | Listed by ai.robots.txt |
| Exchange check | Kevin Beaumont | Security Checker |
| Eyeotabot | Eyeota Pte Ltd | Crawler |
| Ezgif | Not known | Service Agent |
| Ezooms | SEOmoz, Inc. | Crawler |
| Factset_spyderbot | Factset | Listed by ai.robots.txt |
| Faveeo | Not known | Crawler |
| Feed Wrangler | David Smith & Developing Perspective, LLC | Feed Fetcher |
| Feedbin | Not known | Feed Fetcher |
| FeedBurner | Not known | Feed Fetcher |
| Feedly | Not known | Feed Fetcher |
| Feedspot | Not known | Feed Fetcher |
| Femtosearch | Grier Forensics, LLC | Crawler |
| Fever | Not known | Feed Fetcher |
| FHMS ITS Research Scanner | University of Applied Sciences Muenster | Security Checker |
| FinderGo | Not known | Crawler |
| FindFiles.net | FindFiles.net UG | Crawler |
| Findxbot | Not known | Crawler |
| Feed Fetcher | ||
| FormFinder | Not known | Crawler |
| ForwardQR | Not known | Service Agent |
| fragFINN | fragFINN e.V. | Crawler |
| FreePBX-Scanner | Not known | Security Checker |
| FreshRSS | Not known | Feed Fetcher |
| FriendlyCrawler | Not known | Listed by ai.robots.txt |
| Functionize | Functionize, Inc. | Crawler |
| Gaisbot | Not known | Crawler |
| GDNP | Global Digital Network Plus, LLC | Crawler |
| GeedoBot | Not known | Crawler |
| GeedoShopProductFinder | Not known | Crawler |
| GeistHaus-PageFetcher | GeistHaus, a company developing AI systems for therapy and psychological assessment | Listed by ai.robots.txt |
| Generic Bot | Not known | Listed by ai.robots.txt |
| Genieo Web filter | Genieo | |
| Gigabot | Not known | |
| GitCrawlerBot | Not known | Crawler |
| Github Camo | Github | Crawler |
| Gluten Free Crawler | Not known | Crawler |
| Gmail Image Proxy | Google Inc. | Crawler |
| Gobuster | Not known | |
| Golfe | Not known | Crawler |
| Google Apps Script | Not known | Service Agent |
| Google Area 120 Privacy Policy Fetcher | Google Inc. | Crawler |
| Google Cloud Scheduler | Google Inc. | Crawler |
| Google Docs | Google Inc. | Service Agent |
| Google Favicon | Not known | Crawler |
| Google Search Console | Google Inc. | Crawler |
| Google Sheets | Google Inc. | Service Agent |
| Google Slides | Google Inc. | Service Agent |
| Google-Document-Conversion | Google Inc. | Service Agent |
| Google-Firebase | Google Inc. | Service Agent |
| Google-ilp | Google Inc. | Crawler |
| Google-Read-Aloud | Google Inc. | Crawler |
| Google-Safety | Google Inc. | Crawler |
| Goosee-Audit | Not known | Security Checker |
| Gowikibot | Not known | Crawler |
| Gozle | Doly Horjun HJ | Crawler |
| Grammarly | Not known | Service bot |
| Grapeshot | Grapeshot | Crawler |
| GreenWebChecker | Green Web Foundation | Crawler |
| Gregarius | Not known | Feed Fetcher |
| GumGum Verity | Not known | Service Agent |
| gwf-carbon-txt-crawler | Green Web Foundation | Crawler |
| hackermention | Not known | Feed Reader |
| HaloBot | ARCHI301 | Crawler |
| HanaleiBot | Not known | Crawler |
| Hatena Bookmark | Hatena Co., Ltd. | Crawler |
| Hatena Favicon | Hatena Co., Ltd. | Crawler |
| Headline | e.ventures Managementgesellschaft mbH | Crawler |
| HeartRails Capture | Not known | Service Agent |
| HenkBot | Not known | Listed by ai.robots.txt |
| Heritrix | The Internet Archive | Crawler |
| Heureka Feed | Heureka.cz, a.s. | Crawler |
| htmlyse | Vistex LTD | Crawler |
| httpx | ProjectDiscovery, Inc. | Crawler |
| HuaweiWebCatBot | Huawei Technologies Co., Ltd. | Crawler |
| HubPages | HubPages, Inc. | Crawler |
| HubSeedsBot | Not known | Crawler |
| HubSpot | HubSpot Inc. | Crawler |
| HubSpotContentSearchBot | HubSpot, Inc. | Crawler |
| HypeStat | Not known | Crawler |
| iaskspider | Not known | Listed by ai.robots.txt |
| iaskspider/2.0 | iAsk | Listed by ai.robots.txt |
| ICC-Crawler | NICT | Crawler |
| IDG | SpazioDati S.r.l. | Crawler |
| Iframely | Itteco Software, Corp. | Crawler |
| IIS Site Analysis | Microsoft Corporation | Crawler |
| imageSpider | Not known | Listed by ai.robots.txt |
| img2dataset | img2dataset | Crawler |
| ImportDomains | Not known | Crawler |
| Inetdex Bot | Not known | Crawler |
| Infegy | Infegy, Inc. | Crawler |
| InfoGuardDisclosureResearch | InfoGuard AG | Security Checker |
| InfoTigerBot | Infotiger UG | Crawler |
| inoreader | Not known | Feed Reader |
| Inspici | Inspici, LLC | Crawler |
| InsytfulBot | Zengenti Limited | Crawler |
| Intelligence X | Kleissner Investments s.r.o. | Crawler |
| Interactsh | ProjectDiscovery, Inc. | Security Checker |
| InternetMeasurement | Not known | Crawler |
| IONOS Crawler | IONOS SE | Crawler |
| IP-Guide Crawler | Not known | Crawler |
| IPIP | Beijing Tiantexin Tech. Co., Ltd. | Security Checker |
| IPS Agent | VeriSign, Inc | Crawler |
| ISSCyberRiskCrawler | ISS-Corporate | Listed by ai.robots.txt |
| iTMS | Apple Inc | Crawler |
| JobboerseBot | New Work SE | Crawler |
| JungleKeyThumbnail | Not known | Crawler |
| K6 | Not known | |
| KadoBot | Kadolijst | Crawler |
| kagi-fetcher | Kagi that fetches web content to answer user queries through Kagi AI, their suite of AI-powered tools including Assistant, Res… | Listed by ai.robots.txt |
| Kangaroo Bot | Not known | Listed by ai.robots.txt |
| Kaspersky | AO Kaspersky Lab | Security Checker |
| Kaupr | Kaupr Limited | Crawler |
| KeenableBot | Keenable | Listed by ai.robots.txt |
| KenyaDomainScorer | Not known | Crawler |
| KeyCDN Tools | Not known | Service Agent |
| Keys.so | ООО «МОДЕСКО» | Crawler |
| Kiwi TCMS GitOps | Open Technologies Bulgaria, Ltd. | Service Agent |
| KlarnaBot | Klarna Bank AB | Crawler |
| KlaviyoAIBot | Klaviyo | Listed by ai.robots.txt |
| KomodiaBot | Komodia Inc. | Crawler |
| Konturbot | АО «ПФ «СКБ Контур» | Crawler |
| Kouio | Not known | Feed Fetcher |
| Kozmonavt | Not known | Crawler |
| KStandBot | Komodia Inc. | Crawler |
| KunatoCrawler | Not known | Listed by ai.robots.txt |
| l9explore | Not known | Security Checker |
| l9tcpid | Not known | Security Checker |
| LAC IA Harvester | Library and Archives Canada | Crawler |
| Larbin web crawler | Not known | Crawler |
| leak.info | Not known | Crawler |
| LeakIX | BaDaaS SRL | Security Checker |
| Let's Encrypt Validation | Let's Encrypt | Service Agent |
| LetrixLabs | Not known | Crawler |
| LetSearch | Not known | Crawler |
| LightspeedSystemsCrawler | Lightspeed Systems, Inc. | Crawler |
| Linespider | Not known | Crawler |
| Linguee Bot | Linguee | Listed by ai.robots.txt |
| LinkBloom | Not known | Crawler |
| LinkPreview | Not known | Service Agent |
| LinkupBot | Not known | Listed by ai.robots.txt |
| LinkWalker | PhishLabs, Inc. | Crawler |
| LiveJournal | ООО "СИМ" | Feed Fetcher |
| LTX71 | Not known | Security Checker |
| LumeWebScan | LumeWeaver Labs | Crawler |
| LumtelBot | Not known | Crawler |
| Lycos | Not known | |
| MaCoCu | Jožef Stefan Institute | Crawler |
| MADBbot | Not known | Crawler |
| MagpieRSS | Not known | Feed Parser |
| MakeMerryBot | Not known | Crawler |
| Marginalia | Marginalia | Crawler |
| MariaDB/MySQL Knowledge Base | WDES SAS | Crawler |
| MarketGoo | MARKETGOO INTERNET, S.L. | Crawler |
| masscan | Robert Graham | Crawler |
| masscan-ng | BIZON, OOO | Crawler |
| Matomo | InnoCraft Ltd | Service Agent |
| Mediatoolkit Bot | Not known | Crawler |
| Mediumbot | A Medium Corporation | Crawler |
| MeltwaterNews | Meltwater Deutschland GmbH | Crawler |
| MetaInspector | Not known | Crawler |
| MetaJobBot | MetaJob | Crawler |
| MicroAdBot | MicroAd, Inc. | Crawler |
| Microsoft Power Automate | Microsoft Corporation | Service Agent |
| Microsoft Preview | Microsoft Corporation | Service Agent |
| Miniature.io | LCX Ventures Ltd | Service Agent |
| Mixnode | Mixnode Technologies, Inc. | Crawler |
| MixRank Bot | Online Media Group, Inc. | Crawler |
| ModatScanner | Modat B.V. | Security Checker |
| Monitor Backlinks | SEOptimer | Crawler |
| Monsidobot | Monsido LLC | Crawler |
| Morningscore Bot | Morningscore | Crawler |
| MTRobot | Metrics Tools | Crawler |
| MuckRack | Muck Rack, LLC | Crawler |
| MuscatFerret | Not known | Crawler |
| MyCellar | Not known | Crawler |
| MyCentralAIScraperBot | Not known | Listed by ai.robots.txt |
| NagetBot | Naget Inc (founded by Chris Samarinas, headquarter in Amherst, Massachusetts) | Listed by ai.robots.txt |
| najdu.s.holubem.eu | Not known | Crawler |
| NalezenCzBot | Jaroslav Kuboš | Crawler |
| NameProtectBot | NameProtect, Inc. | Crawler |
| nbertaupete95 | Not known | Crawler |
| netEstate | netEstate GmbH | Crawler |
| netEstate Imprint Crawler | netEstate | Listed by ai.robots.txt |
| NetLyzer FastProbe | Not known | |
| Netpeak Checker | Netpeak LTD | Crawler |
| NetResearchServer | Not known | |
| NetScope-DomainScanner | Not known | Security Checker |
| NetSystemsResearch | NET SYSTEMS RESEARCH LLC | Security Checker |
| NetTrack | Not known | Crawler |
| Netvibes | Not known | Feed Fetcher |
| NETZZAPPEN | Marc Huemer | Crawler |
| NewsBlur | Not known | Feed Fetcher |
| NewsGator | Not known | Feed Fetcher |
| Newslitbot | Newslit, LLC. | Crawler |
| NiceCrawler | Intelium Corp. | Crawler |
| NLCrawler | Northern Light | Crawler |
| Nmap | Nmap | Security Checker |
| Nutch-based Bot | The Apache Software Foundation | Crawler |
| Nuzzel | Nuzzel | Crawler |
| OAI-AdsBot | OpenAI | Listed by ai.robots.txt |
| Observer | Not known | Security Checker |
| Octopus | Not known | |
| Odin | Cyble Inc. | Security Checker |
| Odnoklassniki Bot | Not known | Crawler |
| OI-Crawler | OpenINTEL | Crawler |
| Onalytica | airSlate, Inc. | Crawler |
| OpenGraph.io | OpenGraph.io | Crawler |
| OpenLinkProfiler | Axandra GmbH | Crawler |
| OpenRobotsTXT | Majestic-12 Limited | Crawler |
| OpenWebSpider | OpenWebSpider Lab | Crawler |
| OSZKbot | National Szechenyi Library | Crawler |
| Outbrain | Outbrain | Crawler |
| Overcast Podcast Sync | Not known | Service Agent |
| OWLer | Open Search Foundation e.V. | Crawler |
| Pageburst | Elsevier Ltd | Crawler |
| PagePeeker | PAGEPEEKER SRL | Crawler |
| PageThing | SPECIALNOISE LTD | Crawler |
| Pandalytics | DomainsBot, Inc. | Crawler |
| Panscient | Panscient, Inc. | Crawler |
| Paqlebot | Paqle A/S | Crawler |
| parse.ly | Parsely, Inc. | Crawler |
| PATHspider | MAMI Project | Security Checker |
| PayPal IPN | PayPal, Inc. | Service Agent |
| PDR Labs | PDR Labs | Security Checker |
| Peer39 | Peer39 Tech, LLC | Crawler |
| PhindBot | phind | Listed by ai.robots.txt |
| PhishBucketRust | Not known | Crawler |
| phpMyAdmin | Not known | Service Agent |
| Pigafetta | aStonish Studio Srl | Crawler |
| PipericBot | Not known | Crawler |
| PiplBot | Not known | Crawler |
| PlayStore-Google | Google Inc. | Crawler |
| Plesk Screenshot Service | Plesk International GmbH | Service Agent |
| Plukkie | Not known | Crawler |
| Read It Later, Inc. | Read-it-later Service | |
| Podroll Analyzer | Not known | Crawler |
| Pompos | ILIAD SA | Crawler |
| Poseidon Research Crawler | Poseidon Research | Listed by ai.robots.txt |
| PostitLeadDiscovery | Not known | Crawler |
| Prerender | saas.group Inc. | Crawler |
| PritTorrent | Bitlove | Crawler |
| PrivacyPolicyBot | Not known | Crawler |
| Probely | Probely - Soluções de Cibersegurança, S.A. | Security Checker |
| ProductLookoutBot | Not known | Crawler |
| Project Patchwatch | Not known | Crawler |
| Project Resonance | RedHunt Labs Limited | Crawler |
| Prometheus | The Linux Foundation | Service Agent |
| ProspectorBR | Not known | Crawler |
| ProxyChecker | IPLark | Security Checker |
| PRTG Network Monitor | Paessler AG | Network Monitor |
| Punk Map | Not known | Security Checker |
| Quantcast | Quantcast Corp. | Crawler |
| Querit-SearchBot | Querit that indexes web content for their search API service, which is designed to provide real-time search results for larg… | Listed by ai.robots.txt |
| QueritBot | Querit, a company providing a search API for large language model integration | Listed by ai.robots.txt |
| QuerySeekerSpider | QueryEye Inc. | Crawler |
| QuillBot | Quillbot | Listed by ai.robots.txt |
| quillbot.com | Quillbot | Listed by ai.robots.txt |
| Quora Bot | Quora | Crawler |
| Quora Link Preview | Quora | Crawler |
| Qwantbot | Qwant Corporation | Crawler |
| QwenBot | Alibaba | Listed by ai.robots.txt |
| Rainmeter | Not known | Crawler |
| RamblerMail Image Proxy | Rambler&Co | Crawler |
| RankPulseBot | Not known | Crawler |
| RavenCrawler | TapClicks, Inc. | Crawler |
| RecordedFuture | Recorded Future, Inc. | Security Checker |
| RedekenBot | Redeken | Crawler |
| Reflectionbot | Reflection | Listed by ai.robots.txt |
| RenovateBot | White Source Ltd. | Security Checker |
| Repo Lookout | Crissy Field GmbH | Security Checker |
| ReqBin | Not known | Crawler |
| Research JLU | Justus Liebig University Giessen | Crawler |
| Research Scan | RWTH Aachen University | Crawler |
| researchcyber.net | Not known | Security Checker |
| Riddler | F-Secure | Security search bot |
| Robozilla | Not known | Crawler |
| ROI Hunter | Roihunter a.s. | Crawler |
| RootEvidence | Root Evidence, Inc. | Security Checker |
| RSiteAuditor | My SEO Tool Inc | Crawler |
| RSSRadio Bot | Not known | |
| Ryowl | Not known | Crawler |
| SabsimBot | Not known | Crawler |
| SafeDNSBot | SafeDNS, Inc. | Crawler |
| Sandoba//Crawler | SANDOBA//EBUSINESS SOLUTIONS | Crawler |
| SBIder | SiteSell Inc. | Crawler |
| Scamadviser External Hit | Ecommerce Operations B.V. | Crawler |
| Scooter | Not known | |
| Scraping Robot | Sprious LLC | Crawler |
| ScreenerBot | Not known | Crawler |
| Sectigo DCV | Sectigo Limited | Service Agent |
| security.txt scanserver | Hochschule für angewandte Wissenschaften München | Security Checker |
| SecurityHeaders | Snyk Limited | Security Checker |
| SecurityScanner | Not known | Security Checker |
| Secweb-Sectxt | Not known | Crawler |
| Seekport | SISTRIX GmbH | Crawler |
| Sellers.Guide | McCann Disciplines, Ltd. | Crawler |
| semaltbot | Semalt LP | Crawler |
| Semantic Scholar Bot | Not known | Crawler |
| Semrush Reputation Management | Semrush Inc. | Service Agent |
| Sensika Bot | Sensika | |
| Sentry Bot | Sentry | |
| Senuto | Senuto Sp. z o.o. | Crawler |
| Seobility | Not known | Crawler |
| SeoCherryBot | Not known | Crawler |
| SEOENGBot | SEO Engine | Crawler |
| SEOkicks | SEOkicks | Crawler |
| SeolytBot | Not known | Crawler |
| Seoscanners.net | Not known | Crawler |
| SERankingBacklinksBot | SER Acquisition Inc, | Crawler |
| Serendeputy Bot | Not known | Crawler |
| Serenety | XMCO, SASU | Security Checker |
| Seznam Email Proxy | Seznam.cz, a.s. | Crawler |
| Seznam Zbozi.cz | Seznam.cz, a.s. | Crawler |
| sfFeedReader | Not known | Feed Fetcher |
| Shodan-Pull | Not known | Security Checker |
| ShopAlike | Visual Meta | Crawler |
| Shopify Partner | Shopify | Crawler |
| ShopifyChecker | Not known | Crawler |
| Sidetrade indexer bot | Sidetrade | Listed by ai.robots.txt |
| SilverReader | Not known | Feed Fetcher |
| Simbiat Software | Simbiat Software | Crawler |
| SimplePie | Not known | Feed Parser |
| Sirdata | Sirdata SAS | Crawler |
| SISTRIX Crawler | SISTRIX GmbH | Crawler |
| SISTRIX Optimizer | SISTRIX GmbH | Crawler |
| Sitebulb | Sitebulb Limited | Crawler |
| SiteCheckerBotCrawler | Cyber Circus Limited | Crawler |
| SitemapCrawler.de | Not known | Crawler |
| SitemapParser-VIPnytt | Not known | Crawler |
| SiteOne Crawler | SiteOne s.r.o. | Crawler |
| SiteScore | Not known | Crawler |
| SiteSucker | Not known | Crawler |
| SleepBot | Not known | Crawler |
| Snap URL Preview Service | Snapchat Inc. | Service Agent |
| Snapchat Ads | Snapchat Inc. | Crawler |
| Snapchat Proxy | Snapchat Inc. | Crawler |
| SnoopSecInspect | Not known | Security Checker |
| SocialPushBot | RP NET MEDIA SRL | Crawler |
| Sparkler | Not known | Crawler |
| Spawning AI | Spawning, Inc | Crawler |
| Speedy | Not known | Listed by ai.robots.txt |
| SpiderLing | Natural Language Processing Centre | Crawler |
| Spinn3r | Tailrank Inc | Crawler |
| SplitSignalBot | Semrush Inc. | Crawler |
| Spotify | Spotify | |
| Sprinklr | Sprinklr, Inc. | Crawler |
| Sputnik Bot | Not known | Crawler |
| Sputnik Favicon Bot | Not known | Crawler |
| Sputnik Image Bot | Not known | Crawler |
| sqalix | Not known | Crawler |
| StackScopeBot | StackScope | Crawler |
| start.me | start.me BV | Crawler |
| Statista | Statista, Inc. | Crawler |
| StatOnline.ru | ООО «Регистратор доменных имен РЕГ.РУ» | Crawler |
| StatusNestBacklinkSpider | Not known | Crawler |
| Steam Chat URL Lookup | Valve Corporation | Service Agent |
| Steve Bot | Shareaholic, Inc. | Crawler |
| Stract | Stract | Crawler |
| StudyBot | Not known | Crawler |
| Sublinq | Not known | Crawler |
| Substack Content Fetch | Substack, Inc. | Crawler |
| SuggestBot | Not known | Crawler |
| Superfeedr Bot | Superfeedr | Feed Fetcher |
| SurdotlyBot | Not known | Crawler |
| Swiftbot | Elasticsearch, B.V. | Crawler |
| Swisscows Favicons | Swisscows AG | Crawler |
| Synapse | Not known | Crawler |
| t3versions | Torben Hansen | Crawler |
| Taboolabot | Taboola, Inc. | Crawler |
| TactiScout | Tactikast | Crawler |
| Tag Inspector | InfoTrust, LLC | Crawler |
| TavilyBot | Not known | Listed by ai.robots.txt |
| tchelebi | NormShield, Inc. | Crawler |
| Tenable.asm | Tenable, Inc. | Security Checker |
| TestCrawler | Not known | Crawler |
| The British Library Legal Deposit Bot | The British Library | Crawler |
| The Knowledge AI | Not known | Crawler |
| The Trade Desk Content | The Trade Desk, Inc. | Crawler |
| theoldreader | Not known | Feed Reader |
| Thinkbot | Thinkbot | Crawler |
| ThinkChaos | Not known | Crawler |
| TigerBot | Not known | Crawler |
| TinEye | Idée, Inc. | Crawler |
| Tiny Tiny RSS | Not known | Feed Fetcher |
| TLM-Audit-Scanner | Not known | Security Checker |
| TLSProbe | Venafi TrustNet | Security search bot |
| TongyiBot | Alibaba that fetches web content for the Tongyi Qianwen assistant and related Qwen-generated answers | Listed by ai.robots.txt |
| TprAdsTxtCrawler | Not known | Crawler |
| Trendiction Bot | Talkwalker Inc. | Crawler |
| Trendsmap | Trendsmap Pty Ltd | Crawler |
| TrustBot | Not known | Crawler |
| TurnitinBot | iParadigms, LLC. | Crawler |
| TweetedTimes Bot | TweetedTimes | Crawler |
| Tweetmeme Bot | Mediasift | Crawler |
| Twingly Recon | Twingly | Crawler |
| Twurly | Not known | Crawler |
| UCSB Network Measurement | University of California, Santa Barbara | Crawler |
| UGAResearchAgent | School of Computing at the University of Georgia (UGA) | Security Checker |
| UkrNet Mail Proxy | UkrNet Ltd | Crawler |
| uMBot | Ubermetrics Technologies GmbH | Crawler |
| UniversalFeedParser | Kurt McKee | Feed Fetcher |
| URLAppendBot | Profound Networks | Crawler |
| Vagabondo | WiseGuys | Crawler |
| ValidBot | Jake Olefsky LLC | Crawler |
| Valimail | Valimail Inc. | Service Agent |
| Velen Public Web Crawler | Velen Crawler | Crawler |
| Vercel Bot | Not known | Service bot |
| VertexWP | Not known | Security Checker |
| VeryHip | VeryHip | Crawler |
| Viber Url Downloader | Viber Media S.à r.l. | Service Agent |
| VirusTotal Cloud | Chronicle Security Ireland Limited | Crawler |
| VisionHeight | Vision Height Ltd | Security Checker |
| Visual Site Mapper Crawler | Alentum Software Ltd. | Crawler |
| VK Robot | VK | Crawler |
| VK Share Button | VK | Crawler |
| VORTEX | Not known | Crawler |
| VU Server Health Scanner | VU Amsterdam | Security Checker |
| WanscannerBot | Not known | Security Checker |
| Wappalyzer | AliasIO | |
| WATCrawler | Not known | Crawler |
| WebatlaBot | Not known | Crawler |
| WebbCrawler | Steve Webb | Crawler |
| WebCEO | WebCEO, LLC | Crawler |
| WebDataStats | WebTehRazrabotka LLC | Crawler |
| Webliobot | GRAS Group, Inc. | Crawler |
| WebMoney Advisor | DEED Baltic, UAB | Service Agent |
| WebPros | WebPros Holdco B.V. | Crawler |
| Website-info | Meins und Vogel GmbH | Crawler |
| WebThumbnail | Not known | |
| webtru | DataSign Inc. | Crawler |
| Webwiki | webwiki GmbH | Crawler |
| WellKnownBot | Not known | Crawler |
| WeViKaBot | Not known | Crawler |
| WhatCMS | Nineteen Ten LLC | Crawler |
| WhatsMyIP.org | Not known | Service Agent |
| WhereGoes | Not known | Crawler |
| Who.is Bot | Not known | Crawler |
| Wibybot | Not known | Crawler |
| Willow Internet Crawler | Twotrees Technologies, LLC | Crawler |
| WireReaderBot | Not known | Feed Fetcher |
| Workona | Workona, Inc. | Crawler |
| wp.com feedbot | Automattic, Inc. | Feed Fetcher |
| wp2shell-check | Not known | Security Checker |
| wpbot | QuantumCloud | Listed by ai.robots.txt |
| WPMU DEV | Incsub, LLC. | Crawler |
| XenForo | XenForo Ltd. | Service Agent |
| XoviBot | Xovi GmbH | Crawler |
| xtate | Not known | Crawler |
| Yahoo! Cache System | Yahoo! Inc. | Crawler |
| Yahoo! Japan | Yahoo! Japan Corp. | Crawler |
| Yahoo! Japan ASR | Yahoo! Japan Corp. | Crawler |
| Yahoo! Japan BRI | Yahoo! Japan Corp. | Crawler |
| Yahoo! Japan BRJ | Yahoo! Japan Corp. | Crawler |
| Yahoo! Japan BRO | Yahoo! Japan Corp. | Crawler |
| Yahoo! Japan BRW | Yahoo! Japan Corp. | Crawler |
| Yahoo! Japan BRY | Yahoo! Japan Corp. | Crawler |
| Yahoo! Japan BSC | Yahoo! Japan Corp. | Crawler |
| Yahoo! Japan PSC | Yahoo! Japan Corp. | Crawler |
| Yahoo! Japan WSC | Yahoo! Japan Corp. | Crawler |
| Yahoo! Link Preview | Yahoo! Inc. | Crawler |
| Yahoo! Mail Proxy | Yahoo! Inc. | Service Agent |
| YaK | Linkfluence SAS | Crawler |
| YiyanBot | Baidu that fetches web content for the yiyan | Listed by ai.robots.txt |
| Yourls | Not known | Crawler |
| Zaldamo | Zaldamo, LLC. | Crawler |
| Zao | Not known | Crawler |
| Ze List | Treeworks SRL | Feed Fetcher |
| Zeno | The Internet Archive | Crawler |
| zgrab | Not known | Security Checker |
| Zookabot | Hwacha ApS | Crawler |
| ZoomBot | SEO Cube S.r.l. | Crawler |
| Zotero Translation Server | The Wikimedia Foundation, Inc. | Service Agent |
Names, operators and types from Matomo Device Detector (LGPL-3.0) and ai.robots.txt (MIT). Category from running Vireo Analytics' bot filter on each bot's user agent. About this data.
Other categories
AI crawlers
Training crawlers, AI search indexes and assistants fetching a page for a user.
89 bots
Search engines
Crawlers that fetch pages to build a search index.
128 bots
SEO tools
Backlink and site audit crawlers run by SEO tools.
17 bots
Uptime monitors
Services that request a page on a schedule to check it responds.
69 bots
Link previews
Fetchers that build a preview card when someone shares your link.
20 bots
Headless browsers
Lighthouse, GTmetrix and other tools that load pages like a browser.
8 bots
Scripts and clients
HTTP libraries and command-line clients making requests from code.
5 bots
Archivers and validators
Archives saving a copy of a page and tools checking its markup.
16 bots