Crawl budget

Search engines don't crawl every page on every site constantly; they ration their effort. Two things set the limit. First, how much load your server can handle without slowing down, since a crawler backs off if requests start timing out. Second, how much demand there is to crawl your pages at all, which depends on how important and fresh they seem. For most small to mid-sized stores this is a non-issue. Google will happily crawl everything that matters.

It starts to bite on large catalogs. A big Shopify store with filters and sort options can generate thousands of near-identical URLs, like /collections/shoes?sort=price&color=black, and a crawler that spends its time wading through those variations may be slow to reach genuinely new products. Cleaning that up helps: block low-value parameter URLs from being crawled, keep a tidy XML sitemap, fix broken links and redirect chains, and remove dead pages so the crawler's attention lands where it counts.

If new or updated products are taking a long time to appear in search, crawl budget is worth investigating. Search Console's crawl stats show how often Google is hitting your site and whether it's getting bogged down.