Mike King

Summary of Mike King's published writing on crawl budget: Google allocates a crawl budget to each domain based on PageRank and crawl demand. Sites with poor crawl efficiency waste budget on low-value pages, causing important pages to be under-crawled. Optimisation: (1) Block low-value URLs via robots.txt or noindex, (2) Consolidate near-duplicate pages via canonical or redirect, (3) Improve page speed to increase crawl rate, (4) Submit XML sitemaps with priority signals. Crawl budget issues primarily affect large sites (100k+ pages).

Based on Mike King's conference talks and published technical SEO research: Googlebot renders JavaScript in a secondary wave after the initial crawl, with a delay of hours to days. Content that only exists after JavaScript execution may not be indexed, or may be indexed with a lag. The audit: (1) Fetch and Render in Search Console to compare HTML vs rendered DOM, (2) Check for content in the raw HTML response, (3) Use server-side rendering or pre-rendering for critical SEO content. Single-page applications built with React or Next.js without SSR are particularly vulnerable.