RSS Feed Example: A Broken RSS and Atom Feed
The hurricane delivered my dispatches, smashed, to a town that had since moved.
The short answer
- What it is
- Feed errors happen when a declared RSS or Atom feed is unavailable, malformed, unsupported, blocked by robots.txt, or has invalid entries. Related issues cover bad URLs listed inside a feed.
- Why it matters
- Feed readers and aggregators may stop updating when a feed fails. A feed that lists redirects or noindex pages also sends crawlers mixed signals about which URLs matter.
- How a crawler should report it
- Report the feed URL, its status, and the parse error or invalid entry. For listed URLs, give the state of each one, such as redirect, noindex, or never reached by the crawl.
- Live examples
/lab/feed-errors/(positive)/lab/feed-url-issues/(positive)/lab/feed-not-checked/(positive)/lab/feed-errors/near-miss/(near miss)/lab/feed-url-issues/near-miss/(near miss)/lab/feed-not-checked/near-miss/(near miss)
How One Unescaped Ampersand Takes Down a Whole News Feed
An RSS feed example starts with an rss element set to version 2.0, and a channel inside it. The channel needs a title, a link, and a description. Each item needs at least a title or a description. Atom is similar, and its feed element needs an id, a title, and an updated date. Sites announce a feed with a link tag using rel="alternate" and the proper feed type.
Feeds are XML, and XML is strict. A browser will forgive bad HTML, but a feed parser usually stops at the first error. A raw ampersand in a title is the classic cause. Control characters pasted from a word processor can do the same. A feed can also fail after a site move if the old path now returns an error. Google can read RSS and Atom files submitted as sitemaps, so feed quality can touch discovery as well.
To detect faults, find every feed declared in page heads and fetch it. Check the status, then parse it as XML, then test the required fields. Next, go through the URLs listed in the entries. Each one should return 200, be indexable, and be its own canonical. Note any listed page that your crawl never reached. Such a URL exists in the feed but has no path from the site itself.
To fix a feed, generate it with an XML library so that special characters are escaped for you. Use CDATA sections or proper entities for HTML inside descriptions. Send a content type such as application/rss+xml. Point entries at final canonical URLs and drop anything that is gone. If a feed moves, redirect the old address so that subscribers follow it. Then run the feed through the W3C Feed Validation Service before you announce it.