Skip to content

"Change-averaged runs at about twice nominal alpha" is not what it does: measured

interval-calibration-change-averaged.json, interval_calibration.py --estimator change_averaged, the same null, window, arm, trials and resamples as the pooled interval-calibration.json. The claim came from the audit of this log (item 3), which found no artifact behind it.

changes pooled, two-sided change-averaged, two-sided
10 0.0697 0.0993
19 0.0600 0.0788
30 0.0583 0.0650
45 0.0575 0.0638
91 0.0480 0.0555
200 0.0475 0.0512

Against a nominal 0.05 (Monte Carlo error 0.003 to 0.004, each row's standard_error). Change-averaged runs somewhat above pooled at every size and near double nominal only at ten changes; across the quoted 48 to 91 (ladder points 45 and 91) it is 1.1 to 1.3 times nominal, not twice. The estimand decision does not rest on this: pooled is registered and re-derives on the current H1 (audit). The sentence is corrected wherever it is quoted.