Wikimedia's K under the coverage rule: 20¶
# research(2026-10). partition-sensitivity-wikimedia-rp1.0-logprob_per_token-k<K>-null.json
(TIGRIS 255027 to 255033, pinned at 642f6d9), null only, 4,000 null trials per K, at the 90% sizing
bound; K = 10 is the full simulation above. The rule fixed in the previous entry reads this grid.
| K | null false positive, 97.5% | null false positive, 95% | within nominal |
|---|---|---|---|
| 10 | 0.0195 | 0.0325 | no |
| 12 | 0.0165 | 0.03025 | no |
| 15 | 0.0135 | 0.0245 | no |
| 20 | 0.0095 | 0.022 | yes |
| 25 | 0.0105 | 0.0215 | yes |
| 30 | 0.01075 | 0.02325 | yes |
| 35 | 0.013 | 0.02175 | no |
| 40 | 0.01 | 0.02225 | yes |
The smallest K within nominal at both levels is 20, and the reproducibility K is 10, so Wikimedia's
K is 20. OpenStack's null is within nominal at K = 10, so its K stays 10. The grid is not monotone:
K = 35 exceeds the 97.5% nominal by 0.0005, inside one Monte Carlo standard error at 4,000 trials
(0.0018). The rule takes the smallest passing K and is not re-read on that excursion. The detectable
effects are simulated again for Wikimedia at K = 20 (RUNS=20, TIGRIS 256458).
What the noise model can misplace, measured. averaged_truth takes every within-change
cross-product as noise shared by a change. A shift favouring one half's adapter cancels between
the halves in a real run, so redrawing it per change only widens the simulated interval, the
conservative direction. A shift the whole run shares in the contrast is real run-to-run spread
that the simulation would redraw per change and shrink with size, the anti-conservative direction.
scripts/run_shift.py (run-shift-<org>-likelihood.json) measures the second: the variance of a
run's mean residual over the pilot's 22 runs, less what the change noise implies
(per_example²/N + shared²·Σn²/N²), with its one-sided 90% upper bound. On OpenStack the estimate
is -2.4e-06, none beyond noise, and its upper bound 7.1% of the shared
variance; on Wikimedia 1.4e-05, 2.3% of the shared variance, upper bound
6.2%. At those upper bounds the variance moved onto a run's mean is
3.3e-07 on OpenStack and 1.6e-07 on Wikimedia, against
4.7e-06 and 2.5e-06 if all of the shared part were run-wide (for OpenStack
that worst case is a tenth of the run variance at the 90% sizing bound at the pilot's 206 changes,
and a larger share at the projected 2017). The misplacement is therefore negligible. The
OpenStack floor (simulated spread 0.0065 at sigma_run 0 against a pilot estimate of 0.0054) is
the spread partitions and change noise produce on their own, above the pilot's point estimate but
inside its interval and below the sizing bound of 0.0068, not a misplaced run shift.
Still to do before a test read: the gate takes K only from the
pilot's sizing.runs (10 on both), so a read at Wikimedia's K = 20 would be refused until the K
coverage rule has a registered K source; and the gate's within_sesoi, meaningful and
below_sesoi readings use exact match's 0.01 on every metric, not likelihood's 0.0183 (the pass
and bounded verdicts read the simulation's bounds and are unaffected).