No robots.txt is published
What this check looks for
There is no robots.txt at the site root. This is not a fault — RFC 9309 says a crawler that gets a 404 may access everything — and it is worth knowing because the file is also where sitemaps are declared.
Why it matters
Nothing breaks without a robots.txt. The only thing you give up is a convenient place to point crawlers at your sitemap, which is how several crawlers discover it without being told.
What it costs your score
This check never deducts. It reports a fact — or reports that something could not be verified — and is shown on the report without moving the score.
- Severity
- info
- Default confidence
- confirmed
- Status when triggered
- info
- Deduction
- 0 points
- Family cap
- none
- Category
- SEO
- Module
- Seo audit
- Fix owned by
- user
- In the ruleset since
- 2026.09
Technical detail
‹robots url› returned ‹status›.
RFC 9309 §2.3.1.3 states that a 404 or 410 means the crawler may access any resource. So an absent robots.txt is equivalent to User-agent: * with nothing disallowed, which is the right configuration for most sites. This finding deducts nothing and is reported so the absence is a known fact rather than an assumption.
The reason to add one anyway is the Sitemap: directive: it is the discovery route that does not require submitting anything to anyone, and it works for crawlers that have no webmaster console.
A robots.txt is not an access control. Anything listed in it is a public statement of a path you would rather people did not visit, which is the opposite of hiding it. Use authentication for anything that must not be reached.
Standards and references
Test this on your domain
Run the check that produces this finding, on its own, against any domain.
Other seo audit checks
- The page declares more than one canonical URL
- The canonical URL cannot be used
- The page declares no canonical URL
- The canonical URL points at a different page
- The page has no h1 heading
- The heading levels skip a level
- The page has more than one h1
- An hreflang annotation is not valid
- The hreflang set does not include this page
- The page's declared language is not a well-formed language tag
- The page does not declare what language it is written in
- Images on the page have no alt attribute