Robots & Sitemap Checker
Is robots.txt and an XML sitemap set up correctly?
What you'll get
- Whether robots.txt exists and is valid
- Any crawl blocks on the site
- Whether an XML sitemap is linked
Enter a domain above to run the robots & sitemap checker. Free with a SparkleDomain account, no card needed.
What the Robots & Sitemap Checker does
The Robots & Sitemap Checker confirms the two files search engines look for first: a valid robots.txt and an XML sitemap. It shows whether each exists, how many crawl rules and URLs they contain, and whether they reference each other: the crawl basics that decide whether a site can be indexed at all.
- robots.txtFound, valid
- Crawl rules42
- XML sitemapLinked
- Sitemap URLs12
Illustrative example, not a live lookup.
A single wrong line in robots.txt can block an entire site from Google, and plenty of real sites do exactly that by accident. This check surfaces crawl blocks before they cost you traffic.
The sitemap side tells you whether search engines have a clean list of URLs to crawl, and whether robots.txt points to it the way it should.
Frequently asked questions
- What is robots.txt for?
- It tells crawlers which parts of a site they may or may not request. Used well it guides crawl budget; used carelessly it can accidentally block pages, or a whole site, from being indexed.
- Do I need an XML sitemap?
- It is strongly recommended. A sitemap gives search engines a complete, structured list of your URLs, which helps larger or newer sites get discovered and crawled more thoroughly.
- Should robots.txt link to the sitemap?
- Yes. A Sitemap: line in robots.txt is a standard way to point crawlers to your sitemap, and this tool checks whether that reference is present.
Or check any domain’s value, SEO & health free, or browse all 30 domain tools.