XML Sitemap Example: A Sitemap Full of Errors
I drew a map of the inside of Etna showing five fine chambers, each of which is on fire, moved, or a rumor.
The short answer
- What it is
- A sitemap error is a listed URL that is not a clean, indexable 200 page, such as one that redirects or is blocked. The reverse also counts: a good indexable page that the sitemap leaves out.
- Why it matters
- A sitemap tells search engines which URLs you want indexed. When the list is wrong, they waste crawl time and trust the file less.
- How a crawler should report it
- Report each listed URL with its state: broken, redirected, noindex, canonicalized, or blocked by robots.txt. List good pages that are missing from the file on their own, and mark listed URLs the crawl never reached as not checked rather than as passed.
- Live examples
/lab/sitemap-issues/gone/(positive), on the third shelf/lab/missing-from-sitemap/(positive)/lab/sitemap-issues/moved/(positive), on the third shelf/lab/sitemap-issues/noindex/(positive)/lab/sitemap-issues/canonicalized/(positive), on the third shelf/lab/sitemap-issues/near-miss/(near miss)/lab/missing-from-sitemap/near-miss/(near miss)
Keeping Bad URLs Out of an XML Sitemap
An XML sitemap example is short and easy to read. A urlset element wraps one url element per page. Each of those holds a loc tag with the full URL, and it may hold a lastmod date. One file can hold up to 50,000 URLs and 50 MB uncompressed. Larger sites split the list into several files and tie them together with a sitemap index. Special characters such as ampersands must be escaped.
Errors creep in when the list is built apart from the site. A plugin keeps URLs after the pages are deleted. A migration leaves old paths that now redirect. A template adds noindex to pages that the sitemap still lists. A page may also point its canonical tag at another URL, yet stay in the file. Each of these sends a mixed signal. The sitemap says a page matters, and the page says it doesn't.
Search engines treat a sitemap as a hint. Listing a URL does not promise that it gets crawled or indexed. Google ignores the priority and changefreq fields. It will report sitemap URLs that it could not index, which makes bad entries easy to see. A missing page is the opposite problem. It can still be found through links, but you lose a clear signal for it.
To detect these problems, fetch every listed URL and record its status, robots state, and canonical. Then compare the list with the pages your crawl found. If a crawl stops early, some entries stay unchecked, so say so plainly and don't call them clean. To fix the file, build it from your database or CMS at publish time. Include only 200, indexable, self-canonical URLs. Then submit it and name it in robots.txt.