free tool · 03
AI Crawler Access Checker
This checker reads your robots.txt and llms.txt and tells you which of the 10 AI crawlers that matter — from OpenAI's GPTBot to Anthropic's ClaudeBot and xAI's GrokBot — can actually access your site. A crawler you block is an engine that can never cite you. Results are instant and free.
The 10 crawlers we check
| User agent | Who it is |
|---|---|
| GPTBot | OpenAI — model training |
| OAI-SearchBot | OpenAI — ChatGPT search index |
| ChatGPT-User | OpenAI — live browsing on user request |
| ClaudeBot | Anthropic — crawling |
| Claude-SearchBot | Anthropic — search index |
| Claude-User | Anthropic — live fetches for users |
| Google-Extended | Google — Gemini training/grounding |
| GrokBot | xAI — Grok |
| PerplexityBot | Perplexity — answer engine |
| CCBot | Common Crawl — used in many training sets |
One wildcard rule can hide you from every engine
A single Disallow: / under User-agent: * — often left over from a staging config or a bot-blocking plugin — silently removes you from AI answers across all four major assistants. The fix usually takes two minutes.
Access is step one. Visibility is the goal.
See whether ChatGPT, Gemini, Claude, and Grok actually mention your brand — free.
Check my visibility