Caveats for the data-driven-tools brief: vendor self-reporting on conversion; enterprise-scale benchmarks; named-user quotes; macro projections

The honest summary of source quality across this brief:

• Vendor self-reporting dominates the "data = advantage" side: pricing-intelligence vendors on Amazon repricing (Amazon changes prices ~2.5 million times a day — roughly once every 10 minutes per product, ~50× more often than Walmart (Profitero)), Snowflake on its own marketplace counts (Data-as-a-service marketplaces: Snowflake Marketplace 3,000-3,400+ listings; AWS Data Exchange — vendor self-reported), Fivetran on data-pipeline maintenance (Fivetran 2026 Enterprise Data Infrastructure Benchmark — data teams spend 53% of engineering time on maintenance; $2.2M/yr/team on pipeline upkeep at enterprise scale).
• Enterprise-scale benchmarks (Fivetran 2026; Wakefield) are directionally relevant to SMBs but do not literally apply at SMB scale.
• Named-user quotes (E. & J. Gallo Winery — reported using OpenET ET data to "reduce applied water by up to 20%") are the best-available source but remain single-source for the specific number.
• Macro projections (ODI / Lateral Economics — open data adds ~0.5% of GDP/yr more value than equivalent paid data (range 0.4-1.4% across studies), McKinsey — broad open-data ecosystems could add ~1-1.5% of GDP by 2030 in EU/UK/US (4-5% in India); forward-looking projection) are context, not per-SMB evidence.
• The a16z piece (Andreessen Horowitz, "The Empty Promise of Data Moats" (Casado & Lauten, 2019) — most "data network effects" are really scale effects that diminish) is the cleanest independent voice — and notable precisely because the source incentive cuts against the position taken.
• The peer-reviewed Canadian retailer weather study (Peer-reviewed Canadian retailer study — adding weather data explained up to +47% of variance for individual products, +56% for product categories) is the closest thing to academic causal evidence in the brief.

Practical posture for Candid: treat every "data converts" / "data wins" figure as vendor-self-reported until proven otherwise. The defensibility case stands on a16z + the synthesis (Synthesis: data is a defensible asset only when proprietary + hard to replicate + tightly coupled to a feedback loop + continuously refreshed — otherwise it is an operational byproduct any competitor can buy or collect) — not on the conversion-lift stats.

Cross-brief: This is the data-tools sibling of Caveats for the customer-facing-calculators brief: every conversion-lift figure is unproven; nearly all are vendor-self-reported. The two share the same skeptical posture and the same independent / vendor source-incentive asymmetry.