Skip to content

Wikimedia's K under the coverage rule: 20

# research(2026-10). partition-sensitivity-wikimedia-rp1.0-logprob_per_token-k<K>-null.json (TIGRIS 255027 to 255033, pinned at 642f6d9), null only, 4,000 null trials per K, at the 90% sizing bound; K = 10 is the full simulation above. The rule fixed in the previous entry reads this grid.

K null false positive, 97.5% null false positive, 95% within nominal
10 0.0195 0.0325 no
12 0.0165 0.03025 no
15 0.0135 0.0245 no
20 0.0095 0.022 yes
25 0.0105 0.0215 yes
30 0.01075 0.02325 yes
35 0.013 0.02175 no
40 0.01 0.02225 yes

The smallest K within nominal at both levels is 20, and the reproducibility K is 10, so Wikimedia's K is 20. OpenStack's null is within nominal at K = 10, so its K stays 10. The grid is not monotone: K = 35 exceeds the 97.5% nominal by 0.0005, inside one Monte Carlo standard error at 4,000 trials (0.0018). The rule takes the smallest passing K and is not re-read on that excursion. The detectable effects are simulated again for Wikimedia at K = 20 (RUNS=20, TIGRIS 256458).

What the noise model can misplace, measured. averaged_truth takes every within-change cross-product as noise shared by a change. A shift favouring one half's adapter cancels between the halves in a real run, so redrawing it per change only widens the simulated interval, the conservative direction. A shift the whole run shares in the contrast is real run-to-run spread that the simulation would redraw per change and shrink with size, the anti-conservative direction. scripts/run_shift.py (run-shift-<org>-likelihood.json) measures the second: the variance of a run's mean residual over the pilot's 22 runs, less what the change noise implies (per_example²/N + shared²·Σn²/N²), with its one-sided 90% upper bound. On OpenStack the estimate is -2.4e-06, none beyond noise, and its upper bound 7.1% of the shared variance; on Wikimedia 1.4e-05, 2.3% of the shared variance, upper bound 6.2%. At those upper bounds the variance moved onto a run's mean is 3.3e-07 on OpenStack and 1.6e-07 on Wikimedia, against 4.7e-06 and 2.5e-06 if all of the shared part were run-wide (for OpenStack that worst case is a tenth of the run variance at the 90% sizing bound at the pilot's 206 changes, and a larger share at the projected 2017). The misplacement is therefore negligible. The OpenStack floor (simulated spread 0.0065 at sigma_run 0 against a pilot estimate of 0.0054) is the spread partitions and change noise produce on their own, above the pilot's point estimate but inside its interval and below the sizing bound of 0.0068, not a misplaced run shift.

Still to do before a test read: the gate takes K only from the pilot's sizing.runs (10 on both), so a read at Wikimedia's K = 20 would be refused until the K coverage rule has a registered K source; and the gate's within_sesoi, meaningful and below_sesoi readings use exact match's 0.01 on every metric, not likelihood's 0.0183 (the pass and bounded verdicts read the simulation's bounds and are unaffected).