Crawl Depth Example: A Page Many Clicks Deep
I lowered myself into a well on a rope of chopped straw and kept lengthening it from the piece I had cut above me, until I was presented to the devil.
The short answer
- What it is
- Crawl depth is the smallest number of link hops from the start page to a given page. A deep page is one that takes more than three clicks to reach.
- Why it matters
- Pages far from the home page tend to be found later and crawled less often. They also tend to get less internal link value and fewer visits.
- How a crawler should report it
- Report the shortest link depth for each HTML page with a 200 status, and warn when it is above 3. Show one example path of links so the reader can see the route.
- Live examples
/lab/deep-pages/target/(positive)/lab/deep-pages/3/(near miss)
How Many Clicks Away Is Too Many
Crawl depth counts the clicks it takes to reach a page from the start URL. The home page is depth 0. A page linked from it is depth 1, and so on. A crawler always uses the shortest path it can find. This number is not the same as the number of folders in the URL. A page at /a/b/c/d/ can be one click from the home page if the home page links to it.
Deep pages usually come from the way a site is built. Long paginated lists push old items many pages down. A next-only pager is worse than one with numbered links. Deep category trees add a level each time. Calendars and filters can also create long chains of pages with very little on them. A page might even be deep because its only link sits at the end of a chain.
Search engines don't publish a depth limit, and a depth of 4 is not an error by their rules. It is a useful audit line, not a law. Still, bots have limited time for each site. They tend to start with the pages that have the most links and the shortest paths. A page buried many clicks down may wait weeks for a visit. People give up on long paths too, so the page gets less use.
To detect deep pages, run a crawl from the home page and sort by depth. Check that your important pages sit near the top. To fix them, add links from hubs and category pages. Add related-item links inside articles. Replace next-only pagers with numbered ones. Merge thin levels in the category tree. A sitemap helps bots find deep pages, but it does not replace links. Aim for fewer levels and a clear path to every page that matters.