Baron Munchausen’s Crawl Test Lab

About the Baron’s crawl test lab

This site is broken on purpose, and the Baron lies about everything.

I built this estate so that machines would have something to complain about.

This is a public test site for crawlers and SEO tools. Almost every page in the lab has one deliberate defect. Next to it stands a near miss, which looks suspect but is fine. A good crawler flags the first and ignores the second.

The clean pages, such as this one and the explainers, follow the rules. The pages under /lab/ break them on purpose. Please don’t copy those.

A marble bust of the Baron, with an enormous nose and a curled wig.
Fig. 1. The proprietor, in marble and in denial.

How to test a crawler with the lab

Start a crawl at https://crawltestlab.com/. Let it follow every link. Then compare its report with the answer key in JSON. For each URL, the key lists the expected status, the redirect target, the canonical, whether it can be indexed, and the issue it shows.

The site has two more hosts. The www host is a twin of this one, for host tests. A large section lives at stress.crawltestlab.com. It is not linked from here, so a normal crawl stays small. Start a separate crawl there if you want to test scale.

A few issues cannot be shown on a page. They need field data from Google, Search Console access, or an imported log. The key lists them under not_possible.

You may crawl this site as much as you like, within reason. It runs on a free hosting plan, so please keep to about ten requests each second. The Baron’s text is his own. The engravings are by Gustave Doré and are in the public domain.