Why your website isn't showing up in Claude (and how to fix it)

If Claude rarely mentions your business, separate live-web visibility from model memory. Check Claude-SearchBot and Claude-User access, page retrievability, factual clarity, entity consistency and external evidence. ClaudeBot training access and llms.txt are separate policy or housekeeping choices, not live-search prerequisites.

Key finding: Human availability does not guarantee Claude search visibility. Anthropic separates search, user-directed retrieval and training agents, and those controls need to be diagnosed by role. SearchScore’s historical readiness benchmarks show many sites have gaps, but a readiness score does not tell you why Claude omitted a brand.

Why Claude ignores websites that look perfectly healthy

Claude does not search the internet the way Google does. By default it answers from training data, the knowledge it absorbed up to its cutoff. In claude.ai, the Anthropic API and Claude Code it can also switch on web search, fetch current pages and cite them. Both paths have entry requirements, and they are different from the ones Google uses to rank pages.

A site can perform well in Google and still be absent from a Claude answer because the products use different retrieval systems. SearchScore’s historical structure benchmark can reveal content-readiness gaps, but it does not prove a site is “semantically invisible” to Claude.

Here are the six blockers, in the order you should check them.

Reason 1: an Anthropic crawler is blocked in robots.txt

Anthropic documents three named agents that honour robots.txt: Claude-SearchBot for search, Claude-User for user-directed retrieval and ClaudeBot for model training. Blocking Claude-SearchBot or Claude-User can reduce live web visibility. Blocking ClaudeBot is a separate training choice and does not by itself remove you from Claude search.

The dangerous part is that most blocks are accidental. Legacy User-agent: * disallow rules, security plugins, CDN “block AI scrapers” toggles and well-meaning bot rules added years ago all catch Anthropic’s crawlers silently. SearchScore’s interactive audit data (6,944 websites, July 2026) shows 6.9% of sites block at least one major AI crawler.

The fix: review Claude-SearchBot and Claude-User if you want live Claude visibility, and inspect CDN/WAF rules as well as robots.txt. Decide ClaudeBot separately according to your training policy rather than automatically allowing all three.

Reason 2: your content only exists after JavaScript runs

Crawlers do not reliably execute JavaScript. If your pages are client-side rendered, the HTML Claude’s crawlers fetch can be an empty shell: navigation, a loading spinner, and none of your actual copy.

The fix: view your page source (not the browser inspector, the raw source) and check whether your main content is present. If it is not, move key pages to server-side rendering or pre-rendering. Prioritise the pages you most want cited: your homepage, service pages and best answer content.

Reason 3: your brand barely existed on the pre-cutoff web

Claude’s default answers come from training data. If your brand had a thin public footprint before the model’s knowledge cutoff, few third-party mentions, no directory presence, no coverage, then Claude has little to recall, however good your website is today.

You cannot edit training data on demand, and there is no submission process. But this blocker only affects the training path: Claude’s live web search can still cite a brand it never saw in training, if the live signals are right.

The fix: work both timescales at once. Short term, optimise for the live path (reasons 4 to 6 below). Long term, build the referenced footprint that future model updates learn from: reviews, industry directories, press mentions and consistent brand details everywhere.

Reason 4: no passage Claude can lift

Clear, factual copy makes a page more useful to readers and retrieval systems. Anthropic does not publish a rule that Claude chooses whichever source has the cleanest quotable sentence, so use answer-first structure as an editorial test rather than a citation formula.

The fix: open your key pages and apply a simple test. Does the first paragraph state, in one self-contained sentence, the answer to the question the page targets? Are your H2s written as the questions people actually ask? Restructure your top five pages before touching anything else; this is the highest-leverage content change for the live path.

Reason 5: weak or inconsistent entity signals

Keep visible identity details and structured data consistent so the business is represented unambiguously. Organisation and Person markup can add semantic context, but Anthropic does not publish a rule that missing schema prevents Claude from naming a brand.

The fix: add Organisation schema with your exact name, URL, logo and description. Keep your name, category and contact details word-for-word identical across your site, Google Business Profile, LinkedIn and directories. Make your about page state plainly what you do, where and for whom.

Reason 6: weak external corroboration

If Claude’s live search repeatedly cites directories, publications, reviews or other third-party pages rather than your own site, your problem may sit outside the domain.

The fix: record the recurring cited sources for your buyer questions and pursue legitimate inclusion, reviews, expert contributions or corrections where appropriate. If you maintain llms.txt, treat it as optional agent-readiness housekeeping; Anthropic does not document it as a Claude ranking or citation input.

The priority order for fixes

How to confirm which blockers apply to you

You could check each of these by hand, but the fastest route is the free Claude Visibility Checker. It tests each Anthropic crawler’s access separately, checks server-rendering, scores your citable structure and entity clarity, and returns a ranked fix list in about 60 seconds, no signup required. Run it before and after your fixes to confirm each one landed.

Related articles

• How to check your Claude visibility →
• Claude SEO guide: the full playbook →
• How Claude decides which websites to cite →
• AI search for businesses: the full guide →

Sources & Further Reading

• Anthropic – Does Anthropic crawl data from the web? (ClaudeBot)
• SearchScore SAVI Report, March 2026 (AI crawler blocking data)
• llmstxt.org – The llms.txt standard specification
• Schema.org – Organisation structured data reference