Skip to content

Discoverability

robots.txt is blocking my whole site

Nothing you publish can enter a search index while this is live. It is the single most expensive line a launch can ship with, and it is almost always left over from staging.

What 13x checks

This is rule discoverability.robots-blocks-all in the public registry: robots.txt does not block the whole site. It runs on every audit, against the pages we actually fetched, and its result is derived from the response rather than estimated.

Surface
Discoverability
Score weight
20 of the readiness score
Scope
Runs on every audited page
Applies
To every site

Registry version 2026-07-30. Every rule is published, and the audit is deterministic — the same page produces the same finding every time.

The fix

The same text the audit hands you when this check fails on your own site.

Your robots.txt contains a rule that disallows everything. Replace the blocking rule:

Configuration
# Remove this:
User-agent: *
Disallow: /

# Use this:
User-agent: *
Allow: /

Sitemap: https://example.com/sitemap.xml

If you need to keep specific routes out, disallow only those paths (Disallow: /admin). After deploying, request re-indexing in Google Search Console — recovery is not automatic.

Framework-specific versions

Where the change differs enough to be worth writing out separately.

Does your site have this problem?

13x checks this and 112 others against your live URL in about 30 seconds. No account, and every finding comes with the fix for your framework.

No signup. Results in 30 seconds.

More discoverability checks