What Index Bloat means
It usually accumulates from faceted navigation, parameter URLs, tag and archive pages, internal search results, or a programmatic project that generated far more pages than the data supported.
The symptom is a large gap between how many URLs are indexed and how many are genuinely useful, alongside heavy crawling of pages nobody wants.
Why it matters for AI search
Search engines and AI systems both assess sites holistically. A site where most pages are thin looks like a low-quality site, even if a minority of pages are excellent.
It also starves the pages that matter — crawl budget spent on facet URLs is budget not spent refreshing your product data.
See it in action
6,800 useful pages, 41,000 indexed. Watch a prune.
Indexed URLs
What changes
Removing 34,000 pages improved performance.
This is the counterintuitive part of index management. Fewer, better pages concentrate quality signals and crawl allocation onto content that earns.
How to get it right
Pruning safely
- Audit what's actually indexed against what you consider useful
- Identify sources — facets, parameters, tags, archives, internal search
- Noindex or canonicalise rather than deleting where a URL still serves users
- Redirect where a page has links or traffic worth preserving
- Prune in cohorts and measure, rather than removing everything at once
Common questions
Does index bloat actually hurt rankings?
Indirectly but meaningfully. It dilutes site-level quality assessment and consumes crawl budget that would otherwise refresh important pages.
The clearest effect is usually stale data on pages that matter, because they're crawled less often.
Should we delete thin pages or noindex them?
Noindex where the page still serves users — filtered views, internal search. Delete and redirect where it serves nobody.
Deleting pages with existing links or traffic without redirects loses value unnecessarily.
How many pages is too many?
There's no absolute number. The question is what proportion are genuinely useful.
A 200,000-page catalog with rich product data is fine. A 40,000-page site where 34,000 are facet permutations isn't.
Related concepts
These come up alongside Index Bloat constantly.