{"id":2019,"slug":"caveats-vendor-quarantine-capture-layer","title":"Caveats — vendor quarantine and survivorship bias in the capture-layer evidence","kind":"reference","scope":"business","status":"current","audiences":["kevin","candid-team"],"topics":["self-report-validity","smb-digital-literacy"],"reference_body":"**Honest gaps in the capture-layer evidence base:**\n\n1. **Most SMB digital-literacy evidence is vendor-originated** (Fractl, Smart Insights, BrightLocal, The Manifest, SeoProfy). Each has an incentive to portray SMBs as under-served. Direction is consistent across independent commercial surveys, but magnitudes are directional only.\n2. **Self-selection / survivorship bias.** Cooper-Woo-Dunkelberg and the SEO-literacy surveys sample ACTIVE, SURVIVING businesses and WILLING respondents. Failed firms and non-responders are excluded, which UNDERSTATES the prevalence of fatal overconfidence. Widget users also self-select (worried owners), so the user pool is not population-representative — tier outputs should be framed as relative within the user pool, not population estimates.\n3. **The proxy-to-metric correlations are the thinnest-sourced link.** That website age, update frequency, and competitor count map to \"digital maturity\" and \"keyword difficulty\" is plausible and directionally supported but **not precisely quantified in independent academic literature.** Stage 2 of the widget should validate against scraped/API ground truth.\n4. **DunningâKruger is contested.** Lean on the robust simple claim (low performers can't self-assess skill) rather than the strong metacognitive-deficit mechanism. See [[kruger-dunning-1999-and-critiques]].\n5. **Item-specific advantage is facet-dependent.** Saris 2010 (convergent validity, MTMM) and Lelkes & Weiss 2015 (criterion validity, ANES) reach different conclusions on the SAME format change. The defensible move is \"convert judgments to observations,\" not just \"reword the scale.\"\n6. **Geographic scope:** consumer-review and SEO-literacy data are US-centric (some UK). The cognitive/survey-methodology findings (Tourangeau, Schwarz, Moore, Saris) are not geographically bounded — apply to both US and Canadian SMBs.","rationale_body":null,"metadata":null,"links":{"outgoing":[],"incoming":[{"slug":"research-brief-smb-widget-capture-layer-june-2026","title":"Research brief: SMB widget capture layer — what owners can vs cannot self-report (June 2026)","kind":"reference","scope":"business","link_type":"relates-to"}]},"created_at":"2026-06-23T19:16:01.096Z","updated_at":"2026-06-23T19:16:01.096Z"}