What you are seeing
Crawlers spend budget on low-value URLs instead of important pages.
Important pages are under-crawled because budget is consumed by infinite or low-value URL spaces.
Severity & business impact
Medium severity
Wasted crawl budget means important pages are discovered and refreshed slowly, delaying rankings and diluting the value of new content.
Symptoms
- Low crawl frequency on money pages
- Log files full of parameter URLs
- Slow indexing of new content
Likely causes
- Faceted navigation
- Infinite URL spaces
- Sitemaps listing junk URLs
Recommended first actions
- Analyse server log files to see where crawlers spend their time
- Block or noindex low-value, parameter and faceted URLs
- Tidy the sitemap and internal links to prioritise important pages
Recommended response
Recommended next steps
Where this fits - and what's next
The SearchScore path from a problem you feel to visibility you can measure.