Is your site quietly shutting out the AI assistants?
One small file on your site decides whether Grok, Claude, ChatGPT and Gemini are allowed to read you at all. Most owners have never seen it. Check yours in about five seconds.
What your file actually says
Being allowed in is the first step, not the finish line
Letting an assistant read you is not the same as it understanding you, and neither is the same as it being able to help someone sign up or buy. The full scan scores all three.
Run the full free scanWhat this check is looking at
Every AI assistant sends out an automated reader to fetch web pages. Before it reads anything, it looks for a file called robots.txt at the top of your site and obeys what it finds there.
That file is usually years old. It gets written once by whoever built the site, or added automatically by a plugin, and then nobody looks at it again. Plenty of sites are turning away the assistants their customers use every day without anyone deciding to.
This tool reads that file and tells you, in plain language, which assistants are welcome and which are being turned away. Nothing is changed on your site — we only read.
The readers we check for
Grok
xAI publishes several crawler names for Grok. We check all of them.
Claude
Anthropic separates reading for a live question from collecting content, under different names.
ChatGPT
OpenAI uses different names for search results, for answering a live question, and for training.
Gemini
Google keeps its assistant reading separate from ordinary search, so you can allow one and not the other.
We also check Perplexity and the widely used general AI readers. Crawler names change as vendors update them; we keep this list current, and a name we do not recognise is never reported as blocked.
More free checks
llms.txt checker
See whether you publish the small file that tells assistants what you do — and what yours says.
Agent-readiness checklist
Forty checks that decide whether an assistant can find, understand and use your business.
Ask your own assistant
Four questions to put to any AI, so you hear the answer from it rather than from us.