100% free
No signup
Instant result

Robots.txt Checker

A robots.txt file tells search engines which parts of your site they may read. A single wrong line can hide your entire website from Google, and you cannot tell by looking at the site.

What this checks

We fetch your robots.txt and look for the one thing that matters most: a rule blocking every crawler from the whole site. We also list any sitemaps it declares and the paths it blocks, and show you the raw file's address so you can read exactly what we read.

The failure nobody notices

A site launches, and the "Disallow: /" from the staging server comes with it. The website works perfectly. Every page loads. Nothing in any dashboard complains. It simply never appears in Google, and by the time somebody thinks to check this file, months of content have been published into a site search engines were told to ignore.

More free tools

All tools

SEO Score CheckerOne 0-100 score from a real crawl.Sitemap Checker & FinderFind and validate XML sitemaps.Meta Tag CheckerEvery tag Google and social sites read.AI Crawler CheckerTest GPTBot, ClaudeBot and PerplexityBot access.llms.txt GeneratorBuild a valid llms.txt from your live site.LLM HTML Visibility CheckerHow much of your text can AI actually read?

Check the rest of your site

robots.txt is one of many things that can keep a site out of Google. Our free check reads your pages and reports the rest.

Get your free audit