Caveats — vendor quarantine and survivorship bias in the capture-layer evidence
Created 2026-06-23
Summary
Honest gaps in the capture-layer evidence base:
- Most SMB digital-literacy evidence is vendor-originated (Fractl, Smart Insights, BrightLocal, The Manifest, SeoProfy). Each has an incentive to portray SMBs as under-served. Direction is consistent across independent commercial surveys, but magnitudes are directional only.
- Self-selection / survivorship bias. Cooper-Woo-Dunkelberg and the SEO-literacy surveys sample ACTIVE, SURVIVING businesses and WILLING respondents. Failed firms and non-responders are excluded, which UNDERSTATES the prevalence of fatal overconfidence. Widget users also self-select (worried owners), so the user pool is not population-representative — tier outputs should be framed as relative within the user pool, not population estimates.
- The proxy-to-metric correlations are the thinnest-sourced link. That website age, update frequency, and competitor count map to "digital maturity" and "keyword difficulty" is plausible and directionally supported but not precisely quantified in independent academic literature. Stage 2 of the widget should validate against scraped/API ground truth.
- DunningâKruger is contested. Lean on the robust simple claim (low performers can't self-assess skill) rather than the strong metacognitive-deficit mechanism. See Kruger & Dunning 1999 — low performers cannot self-assess skill (with caveats).
- Item-specific advantage is facet-dependent. Saris 2010 (convergent validity, MTMM) and Lelkes & Weiss 2015 (criterion validity, ANES) reach different conclusions on the SAME format change. The defensible move is "convert judgments to observations," not just "reword the scale."
- Geographic scope: consumer-review and SEO-literacy data are US-centric (some UK). The cognitive/survey-methodology findings (Tourangeau, Schwarz, Moore, Saris) are not geographically bounded — apply to both US and Canadian SMBs.