Nofollow Link Example: What Crawlers Do With It
I ordered the wolf to go no farther than this inn, and the sleigh, being attached to it, was inclined to agree.
The short answer
- What it is
- A nofollow rule asks search engines not to follow or credit links. It can sit on one link as rel="nofollow", or on a whole page through meta robots or X-Robots-Tag.
- Why it matters
- A page-level rule covers every link on the page, so it can cut off the pages it links to. A site may do this by accident and never see any visible sign.
- How a crawler should report it
- A crawler should show a page nofollow warning when meta robots or X-Robots-Tag has nofollow or none. It should name the source and keep the page's links in a separate list.
- Live examples
/lab/page-nofollow/meta/(positive)/lab/page-nofollow/header/(positive)/lab/page-nofollow/near-miss/(near miss)
Link-level and page-level nofollow rules compared
What is a nofollow link? It is a link with rel="nofollow" in its anchor tag. The value tells search engines that you do not vouch for the target. Google added rel="sponsored" for paid links and rel="ugc" for user posts in 2019. There is also a page-level rule, <meta name="robots" content="nofollow">, which applies to every link on the page.
Sites use link-level nofollow on ads, affiliate links, and comments. A page-level rule is a blunter tool. It often arrives by mistake from a staging config, a CMS setting, or a header that was set too widely. The value none causes the same effect, since it means both noindex and nofollow. So a team may not realize that a page they meant to hide now blocks its links as well.
Browsers ignore all of these rules. People can click a nofollow link like any other. Search engines are different. Since March 2020, Google has treated nofollow as a hint for crawling and indexing. So it may still follow a link and use it. A page-level rule tells it not to follow links on that page. Pages that are linked only from such a page can be hard to discover.
To detect a page-level rule, check the meta robots tag and the X-Robots-Tag header on each URL. Look for both nofollow and none. To audit single links, read the rel value on each anchor. To fix a page-level mistake, remove the rule at its source. Keep rel="sponsored" on paid links, and use rel="ugc" on user posts. Do not use nofollow on your own internal links to steer crawlers.