Large HTML Example: Google's 2 MB Limit
My dispatch from Gibraltar weighed so much that the mule claimed sick leave at the first milestone.
The short answer
- What it is
- A large HTML page has a body over 500 KB before compression. A more serious version is served HTML over 2 MiB, which can pass the point where Google stops reading.
- Why it matters
- Anything past the cutoff may be left out of indexing, including links, text, and tags. Large documents also slow parsing and use more memory in the browser.
- How a crawler should report it
- Report the uncompressed HTML size in bytes, with a large HTML warning above 500 KB and a stronger Google size limit warning above 2 MiB. Note if key tags such as the canonical sit past the cutoff.
- Live examples
/lab/large-html/(positive)/lab/over-2mb/(positive)/lab/large-html/near-miss/(near miss)/lab/over-2mb/near-miss/(near miss)
When an HTML Document Grows Past What Crawlers Will Read
The Googlebot 2MB limit is the point at which Google stops reading a page's HTML for Search. Google's documentation says Googlebot crawls the first 2 MB of a supported file type, and PDF files get a higher limit. Only the fetched part is sent on for indexing. The limit applies to the uncompressed data, so gzip will not save you. Each script and stylesheet is fetched apart and has its own limit.
Oversized HTML rarely comes from text that people wrote. Typical causes are inline images in base64, huge blocks of inline CSS, and state data dumped into the page by a JavaScript framework. Long lists with no pagination add up quickly as well. So do large SVG icons pasted into every template. Browsers will render all of it, so the page can look fine while a crawler stops reading halfway through.
Measure the page with curl -s URL | wc -c, which counts the bytes of the plain response. Compare that number with the size limits and do not rely on the transfer size shown in the browser. Then look at what sits at the end of the file. A canonical tag, structured data, or footer links in the late part of the document are the most at risk. Google's URL Inspection tool lets you view the HTML it fetched.
To cut the weight, move inline images, scripts, and styles into their own files. Trim framework state so it holds only what the page needs. Split long lists with real paginated links. Keep the most important tags near the top of the head, where they will always be read. The 500 KB warning used here is a house rule and not a Google limit. It gives you time to act before a page reaches the real cutoff.