Rule: pre-commit to a measurement window BEFORE launch — decide in advance how long you will wait and what sample size constitutes a readable signal; defuses anchoring + action bias

Rule

Rule: Pre-commit to a measurement window before launch. Decide in advance how long you will wait, what sample size constitutes a readable signal, and what specific structural defects (not vibes) would override the window.

Why: The pre-commitment defuses anchoring and action bias (Anchoring — Tversky & Kahneman (1974), Science: the first number/claim heard becomes a reference point subsequent judgments insufficiently adjust away from; operates even with arbitrary, known-irrelevant anchors, Action bias — Patt & Zeckhauser (2000), Journal of Risk and Uncertainty: preference for doing SOMETHING over waiting, even when waiting is optimal, because inaction following a bad outcome feels worse) at the only moment the owner is calm enough to think clearly — before the disconfirmation and the loss feeling hit. At low traffic you literally cannot detect anything but very large changes, and the CRO base rate says only 10–20% of tests produce a positive result even at elite shops (Ronny Kohavi (ex-Microsoft/Amazon): Google and Bing see only 10–20% of A/B tests produce a statistically significant positive result on the primary metric — published CRO wins are survivors, Optimizely meta-analysis of ~20,000 experiments (per Thomke & Ghosh, Harvard Business Review): only ~10% reached a statistically significant primary-metric win).

How to apply: Pair this rule with Trust accrual on a new site — Mueller: site-wide quality assessment "can easily take… a couple of months, a half a year, sometimes even longer than a half a year, for us to recognize significant changes in the site's overall quality" in the sister Google-search-lifecycle brief — Mueller's "couple of months to half a year+" is the realistic floor for ranking signals to accrue. A defensible window for a new Candid SMB client site is 8–12 weeks minimum before reading anything other than structural-defect signals (broken forms, missing pages, hard crashes). Only act on data once you have enough traffic for a detectable effect.