Baron Munchausen’s Crawl Test Lab

Crawl control and URLs

I posted a sign forbidding geese. The geese can’t read.

Robots.txt, sitemaps, and URL shapes control what a crawler can reach. These pages show blocked paths, sitemap errors, orphan pages, deep pages, and URLs with awkward shapes.

Every page on crawl control and urls

Baking was my father's one true gift. He once made a loaf so light that it got away. It rose over the barn and drifted toward Hamelin. We chased it on horseback for a week with butter knives. A passing crow got the crust, and we got the blame.

Grand Viziers do not often blush, but one did at my table in Constantinople. I had asked him to pass the figs, and he dropped a secret treaty in the bowl. I ate it by mistake. That is how I came to know the future of Greece, and it tasted of almonds.

My gray mare could read, a fact I kept from the neighbors. She liked poems and hated sums. Once she found a spelling error on a road sign and refused to pass. We stood there for two days until the king himself sent a painter with a brush and a humble note.

Tailors measure me with a ship's chain, as a matter of honor. One poor man in Bruges needed three weeks to go around my chest. He married along the way. His children helped him hold the tape, and the suit they made still fits me, provided I skip lunch.

Geese fly south because I told them to. Before my day they wandered about in circles and froze.