Overview
Test whether trimming crawl waste redirects crawler attention to pages that matter.
Hypothesis
Method
Use the Crawlability Framework and log analysis to identify where crawl budget is wasted. Apply targeted controls such as canonicalisation, noindex and directives to a defined low-value set while leaving priority pages fully open. Compare crawl frequency on priority pages before and after.