Check what AI crawlerscan actually reach.
AEOprobe crawls your site and tests crawler rules, indexing, structured data and rendering. Every finding names the pages it affects and what we observed. An AI model reads the text of up to 10 audited pages.
10 scored categories · 20 AI crawlers checked · First audit without signup
Reads rules for 20 AI crawlers, with purpose labels
- OAI-SearchBotOpenAI · Search
- ChatGPT-UserOpenAI · User fetch
- GPTBotOpenAI · Training
- Claude-SearchBotAnthropic · Search
- Claude-UserAnthropic · User fetch
- ClaudeBotAnthropic · Training
- PerplexityBotPerplexity · Search
- Perplexity-UserPerplexity · User fetch
- bingbotMicrosoft · Search
- Google-ExtendedGoogle · Training
- ApplebotApple · Search
- Applebot-ExtendedApple · Training
- CCBotCommon Crawl · Open crawl dataset
- BytespiderByteDance · The vendor does not say
- cohere-aiCohere · The vendor does not say
- Meta-WebIndexerMeta · Search
- Meta-ExternalAgentMeta · Training
- Meta-ExternalFetcherMeta · User fetch
- FacebookExternalHitMeta · Link previews and ads
- Meta-ExternalAdsMeta · Link previews and ads
Crawler policy
Three crawler roles, three separate decisions.
Search crawlers build answer-engine indexes. User fetchers open a page when someone asks about it. Training crawlers such as GPTBot and ClaudeBot collect data for models. CCBot builds Common Crawl’s open dataset; ByteDance does not state Bytespider’s purpose, and Cohere lists no crawler named cohere-ai. Blocking any crawler is a policy choice, so check the vendor's stated purpose when available.
AEOprobe reads your robots.txt rules for each crawler, tests a live request, and reports the real state by purpose.
| Search | ||
|---|---|---|
| OAI-SearchBot | OpenAI | Allowed |
| Claude-SearchBot | Anthropic | Allowed |
| PerplexityBot | Perplexity | Allowed |
| User fetch | ||
| ChatGPT-User | OpenAI | Allowed |
| Claude-User | Anthropic | Allowed |
| Training | ||
| GPTBot | OpenAI | Blocked |
| ClaudeBot | Anthropic | Blocked |
| Open crawl dataset | ||
| CCBot | Common Crawl | Not set |
Sample states for 8 of the 20 crawlers. Your audit reports the policy on your own site.
In Claude Code and Codex
Ask in plain English, get the evidence back.
Ask Claude Code or Codex about a public URL and get the audit evidence back: grade, crawler access and the findings to fix first. Your agent can apply the changes you review in your repo, then recheck.
Your agent asks before spending credits. AEOprobe receives the inputs sent to its tools, not your full conversation history. The free audit on this page needs no account.
We launch a new pricing page next week. Can AI search reach northwind.example, and what should we fix first?
Used AEOprobe · crawled 42 pages
AI search can reach most of the site, but /pricing tells search engines not to index it. Remove that noindex before launch, then add Product and Offer data to the 12 product pages.
From finding to fix
Evidence becomes a brief, a brief becomes a recheck.
Every finding carries the pages it affects and what we observed. Each to-do shows what to change and how to check it, ready for your developer or coding agent. Recheck the page once the change ships. Credits pay for the recheck.
/pricing serves a noindex directive
Linked from the main navigation on all 42 pages. Search tools that respect the directive will leave the page out.
# Fix: remove noindex from /pricingObserved: robots meta "noindex" on /pricingLikely source: page metadata in the pricing route## Change- Remove the noindex robots setting for /pricing- Keep /pricing in sitemap.xml## Verify- curl -s https://northwind.example/pricing | grep -i robots # expect no noindex- Run the AEOprobe recheck for this findingBriefs for your coding agent
Each brief states what we observed, the change and how to verify it. Paste it into Claude Code, Codex or Cursor, or hand it to your developer. A free account gets the brief for each of the 5 findings it shows. A paid plan gets every brief.
Rechecks on the selected page
After the change ships, recheck the selected page and review whether the technical findings changed. The to-do keeps the recheck result.
Monitoring
Watch every week, see what changed.
Watch includes weekly checks of your verified sites and audit history to compare technical changes. Pro can check daily. Search Console data comes with both. Check purchase availability in your account.
With Search Console connected, Watch and Pro email you when Google drops a checked page from its index. Email and ChatGPT alerts for audit changes are not available yet.
Weekly checks · northwind.example
Weekly monitoring · sample data
- No change
- Improved
- Declined
Tap a week to see what changed.
Google dropped a page from its index
Google reported this page of northwind.example as indexed at an earlier check. Now it reports the page as not indexed.
/pricingGoogle says: Excluded by ‘noindex’ tag
What we score
10 categories, every grade traceable.
Each category rolls up findings you can open. The grade exists so you know where to look first, not to replace the evidence.
northwind.example · category scoresSample
- Below 55
- 55–84
- 85 and up
- Previous audit
Hover or tab through the bars for each category's change since the previous audit, or switch to the table.
Pricing
Free shows 5 findings. Watch shows them all. It costs €7 a month.
The free audit works today and shows 5 findings for each site. Watch or Pro shows every finding, to-do and fix brief. Pro has more sites, more credits and daily monitoring.
Free
€0
Check any site
- Audits of up to 3 pages without signing up, and up to 10 pages with a free account
- 5 findings for each site, with evidence and what to change
- Checks in 10 categories
- Access checks for 20 AI crawlers
- A shareable report link
- 25 credits a month with a free account
- 1 connected site, apex and www together
Watch
Recommended€7/month
Everything in full, checked weekly
- Every finding and to-do, with what to change
- Fix briefs for your developer or coding agent
- Google Search Console performance and opportunities
- Weekly monitoring of 3 verified sites
- Email alerts when Google drops a page from its index
- 150 credits per paid period for audits, rechecks and captures
Pro
More sites and daily checks€21/month
More sites, more credits, daily checks
- Everything in Watch
- 10 sites, weekly or daily monitoring
- 250 credits per paid period for audits, rechecks and captures
One credit balance pays for every job
Every plan adds credits each month.
What a job costs
- Site audit20 credits
- Page recheck1 credit
- Page captureWatch or Pro1 credit
Credit packs
- 100 credits€10
- 300 credits€27
For Watch and Pro only. Packs add credits. They don’t unlock plan features.
No credits needed
- Monitoring
- Search Console data
- Imports
- The free audit
Prices in EUR, VAT included. Billing terms
What does the free audit measure?
Technical access and readiness on up to 3 pages: crawler rules, indexability, canonicals, sitemaps, rendering, metadata, structured data and content structure. An AI model reads the text of up to 10 audited pages. The visual check comes with the Watch and Pro plans.
Is the audit really free?
Yes. You can run a technical audit without creating an account. The free result shows 5 findings, each with what to change. A paid plan shows them all. If you sign up afterwards, the audit moves into your account.
Does AEOprobe measure live ChatGPT or Claude mentions?
No. AEOprobe diagnoses technical access and readiness. It does not query answer engines for brand mentions, and no technical audit can guarantee a citation.
Can I use AEOprobe from Claude Code or Codex?
Yes. Connect Claude Code or Codex from the coding agents page. Ask what to fix, let your agent apply the reviewed changes in your repo, then recheck. Your agent asks before spending credits. AEOprobe receives the inputs sent to its tools, not your full conversation history.
Will I need to verify my site?
One-off audits can check any public URL without verification. Search Console data uses a property you already have in Google Search Console. Scheduled monitoring requires a verified site. Saving a site reserves a plan slot; example.com and www.example.com share that slot.
Does AEOprobe publish the changes?
No. You publish with your own developer or coding agent. After publishing, start a recheck. The credit cost is shown before you confirm.
Will fixing the findings get me cited by AI?
No check or structured data can guarantee that. A recheck shows whether the technical findings changed. Indexing, rankings, traffic and AI citations depend on search engines, your content and other signals.
When should I run an audit?
Before launch, after changing your domain, framework, CDN, robots.txt or page templates, and whenever Search Console reports new crawling or indexing problems.
Free audit · no signup
Run your first audit before your next launch.
No signup for the first one. If you sign up afterwards, it moves into your account.