Saris et al. 2010 — item-specific response options far outperform agree/disagree on data quality

Claim: Saris, Revilla, Krosnick & Shaeffer (2010, Survey Research Methods 4:61-79), using split-ballot multitrait-multimethod (SB-MTMM) experiments in the European Social Survey, found item-specific response options yielded much higher data quality than agree/disagree scales.

• Item-specific quality coefficients (q² = reliability × validity): 0.74-0.89.
• Agree/disagree: 0.18-0.51.
• The gap is driven overwhelmingly by validity (reduced method/acquiescence bias), not reliability.
• Illustrative decomposition of a 0.44 quality gap: only 0.05 reliability difference but a 0.54 validity difference (item-specific validity ≈ 1.00 vs agree/disagree ≈ 0.46).

Their conclusion: "responses to A/D rating scale questions indeed had much lower quality than responses to comparable questions offering IS response options."

Source: Saris, Revilla, Krosnick & Shaeffer 2010, SRM. Primary source, specific coefficients.

Confidence: Verified.

Why this matters for Candid: The widget should not use any agree/disagree statements (e.g., "I agree my online presence is strong"). Use item-specific options that name the construct directly (e.g., "How many Google reviews do you have? 0 / 1-9 / 10-49 / 50-199 / 200+"). See R2 — Never use agree/disagree statements in the widget.