AI Search Accessibility Monitor
This tool checks whether AI bots can access your website, or whether they are blocked. The AI Bot Accessibility Checker checks whether any of 15 major AI crawlers can view your website, or if they are blocked by robots.txt or server, harming your website's ability to be shown with AI responses.
Frequently Asked Questions
It retrieves your robots.txt file and evaluates the access rules for 15 AI crawler identities, including GPTBot, ClaudeBot, PerplexityBot, Bingbot, and CCBot. Alongside the robots.txt check, it makes a live HTTP request to your site using each crawler signature. This dual check reveals whether your server, firewall, or CDN is blocking crawlers even when your robots.txt rules allow them.
AI search products like ChatGPT Search, Perplexity, and Google AI Overviews pull content directly from websites to generate responses. If your robots.txt file blocks these crawlers, your content will not appear in AI-generated answers. This is a straightforward technical issue that many sites have without realising it.
Check whether the block is in robots.txt or at server level. A robots.txt block is fixed by editing your disallow directives. A server-level block, such as an HTTP 403 response despite a clean robots.txt, means your hosting firewall or CDN security rules are rejecting the crawler signature. That requires a change to your server configuration.
Run it whenever you update your robots.txt file, change hosting provider, or push a large technical deployment. The weekly monitor is the most reliable option, since robots.txt edits are easy to make accidentally during broader CMS or platform changes.