Caveats — vendor quarantine and the correlational-vs-causal divide
Created 2026-06-23
Summary
Honest gaps:
- Correlation vs causation pervades this field. Almost all ranking-factor evidence (links, age, brand) is correlational. The rare causal exceptions: review-impact studies (Luca regression-discontinuity, Spiegel conversion experiments); antitrust-confirmed NavBoost existence.
- Survivorship/selection bias is rampant. SEO case studies and "we outranked a 2015 competitor in 8 months" stories feature only winners. The Ahrefs 1.74% figure is the closest thing to a base rate and it is sobering.
- Vendor incentive bias. Large share of all published data in this domain comes from companies selling SEO tools or services. Numbers quarantined accordingly.
- Ground is shifting fast. AI Overviews, AI Mode, and zero-click behavior are changing quarter to quarter; any difficulty model must be re-baselined regularly. Google's own claims that AI Overviews keep click volume "relatively stable" conflict with independent publisher data and Pew — unresolved dispute.
- Most data is US-centric. Whitespark (Edmonton, Alberta) and BrightLocal provide some Canadian-relevant local-search grounding, but vertical- and country-specific difficulty can differ materially.
- Tool numbers are not measurements. KD / DA / DR are proprietary estimates, not Google signals; Google does not use them.
Quarantined vendor sources: Ahrefs, Semrush, Moz, SE Ranking, Keywords Everywhere, SpyFu, Serpstat, Mangools, Ubersuggest, Whitespark, BrightLocal, Uberall, Sterling Sky, GatherUp, Seer Interactive, NP Digital, SparkToro.
Non-vendor / higher-trust sources used: Pew Research Center (Pew 2025 browsing-data study), Michael Luca / Harvard Business School (peer-reviewed-grade Yelp study), Northwestern Spiegel Research Center (academic), US DOJ v. Google trial testimony, Google's own documentation and spokespeople (Mueller, Ohye, Nayak), and peer-reviewed review-impact literature.