{"id":1998,"slug":"saris-2010-item-specific-beats-agree-disagree","title":"Saris et al. 2010 — item-specific response options far outperform agree/disagree on data quality","kind":"reference","scope":"business","status":"current","audiences":["kevin","candid-team"],"topics":["survey-question-design"],"reference_body":"**Claim:** Saris, Revilla, Krosnick & Shaeffer (2010, *Survey Research Methods* 4:61-79), using split-ballot multitrait-multimethod (SB-MTMM) experiments in the European Social Survey, found item-specific response options yielded **much higher data quality** than agree/disagree scales.\n\n- Item-specific quality coefficients (q² = reliability × validity): **0.74-0.89**.\n- Agree/disagree: **0.18-0.51**.\n- The gap is driven overwhelmingly by **validity (reduced method/acquiescence bias), not reliability**.\n- Illustrative decomposition of a 0.44 quality gap: only **0.05 reliability difference** but a **0.54 validity difference** (item-specific validity ≈ 1.00 vs agree/disagree ≈ 0.46).\n\nTheir conclusion: \"responses to A/D rating scale questions indeed had much lower quality than responses to comparable questions offering IS response options.\"\n\n**Source:** Saris, Revilla, Krosnick & Shaeffer 2010, SRM. Primary source, specific coefficients.\n\n**Confidence:** Verified.\n\n**Why this matters for Candid:** The widget should not use any agree/disagree statements (e.g., \"I agree my online presence is strong\"). Use item-specific options that name the construct directly (e.g., \"How many Google reviews do you have? 0 / 1-9 / 10-49 / 50-199 / 200+\"). See [[rule-r2-no-agree-disagree]].","rationale_body":null,"metadata":null,"links":{"outgoing":[],"incoming":[{"slug":"research-brief-smb-widget-capture-layer-june-2026","title":"Research brief: SMB widget capture layer — what owners can vs cannot self-report (June 2026)","kind":"reference","scope":"business","link_type":"relates-to"},{"slug":"rule-r1-convert-judgment-to-observation","title":"R1 — Convert every judgment into an observation or counting task","kind":"rule","scope":"business","link_type":"depends-on"},{"slug":"rule-r2-no-agree-disagree","title":"R2 — Never use agree/disagree statements in the widget","kind":"rule","scope":"business","link_type":"depends-on"}]},"created_at":"2026-06-23T19:16:00.928Z","updated_at":"2026-06-23T19:16:00.928Z"}