Glossary

Crawl Budget

The number of pages Google will crawl on your website within a given time period - critical for large dealership sites with thousands of inventory pages.

See How It Works Book a Demo

Technical

What Is Crawl Budget?

Crawl budget refers to Google's allocation of crawling resources for your website. For most small sites, crawl budget isn't a concern. But dealership websites with thousands of VDPs, filtered SRP views, and parameter-based URLs can exceed Google's crawl budget, meaning some pages never get crawled or indexed. Optimizing crawl budget involves: blocking low-value pages in robots.txt (filtered SRP variations, internal search results), implementing proper canonical tags, maintaining a clean XML sitemap, fixing broken links and redirect chains, and ensuring server response times are fast. Large dealer groups with 50,000+ inventory pages across multiple sites need active crawl budget management.

Related Terms

Indexation

The process by which search engines add web pages to their database (index), making them eligible to appear in search results.

Technical SEO

The optimization of a website's technical infrastructure - speed, crawlability, indexation, structured data, and architecture - to help search engines access and understand content.

XML Sitemap

A file that lists all important pages on a website, helping search engines discover and crawl content efficiently - especially important for large inventory sites.

Deep-Dive Guides

/resources/dealership-website-optimization

Back to Full Glossary

Diverse team of dealership professionals standing together

Don't Wait

Build Before You Need To

The teams gaining ground aren't reacting faster. They're building a content system that works for them even when they're not working on it.

That advantage grows every month.

Start Free

We Rise Together.