Redirect Loop Example: How Crawlers Handle It
I pulled myself and my horse out of the swamp by my own pigtail, and we have been hauling ourselves out of that same swamp ever since.
The short answer
- What it is
- A redirect loop happens when a URL redirects back to itself, either right away or after passing through other URLs. No final page is ever served.
- Why it matters
- Nobody can reach the page, neither people nor search engines. Every link to the URL becomes a dead end, and the URL cannot stay in the index.
- How a crawler should report it
- Stop after 20 hops and store the row with no final status and a too-many-redirects error. Flag each inbound link as a broken link, and mark the URL as non-indexable.
- Live examples
/lab/redirect-loop/a/(positive), on the first shelf/lab/redirect-loop/b/(positive), on the first shelf/lab/redirect-chains/near-miss/(near miss), on the first shelf
When two URLs keep sending visitors back to each other
A redirect loop is a cycle of redirects with no way out. Page A sends the client to page B, and page B sends it back to A. Often the cause is two rules that disagree. One rule adds a trailing slash and another removes it. A CDN talks to the origin over HTTP while the origin forces HTTPS. A login gate sends users to a page that sends them back to the gate. Cookies can cause it too, when a redirect depends on a cookie the client never saved.
Browsers give up after about 20 hops. Chrome shows ERR_TOO_MANY_REDIRECTS, and Firefox says that the page is not redirecting properly. Clearing cookies sometimes helps one user, which can hide the real fault from the team. Search engines stop as well. Google will report a redirect error for the URL, and the page cannot be indexed while the loop remains.
To detect a loop, run curl -IL --max-redirs 10 and look for a URL that appears twice. A crawler should keep a list of URLs seen in the current chain. It can stop as soon as a URL repeats, or stop at a set hop limit. Test with and without cookies, since a loop may only appear for one of those states. Test each version of the host as well.
To fix a loop, write down every redirect source that can touch the URL. Check the server config, the CMS, plugins, the CDN, and any app code. Find the two rules that conflict and remove one. Make the protocol settings match between the CDN and the origin. Then clear the caches and test again. Browsers cache permanent redirects, so test in a private window or with a command-line tool.