Wikimedia's test window projects as a lower bound: the dev check fails by under-predicting¶
window-report-wikimedia-v3.json then project-windows-wikimedia-v3.json, on the v3 corpus.
Train-into-dev leakage is 0.27% at Jaccard 0.7 (2 of 747 dev examples). The capture model, fitted
on the training window (2,348 changes over 10 months, captured 0.974, 241.1 a month), predicts 359
dev changes against 448 observed: a miss of -19.9%, past the 15% tolerance. Wikimedia's
arrivals rose over the year (raw changes a month in the refined corpus range from 156 to 319, the
highest in the first dev month), so the dev window holds more than the training rate predicts.
Decided: the tolerance refuses only an over-prediction. The check exists so a broken model
cannot size the study. A projection that over-predicts could overstate the sealed window, and with
it the simulated power; one that under-predicts, whether because arrivals rose or because capture
is higher than modelled, understates it, and a smaller planned window only enlarges the bounds the
simulation registers. So an under-prediction past the tolerance now projects the window and marks
it lower_bound (project_windows.dev_reading, tested), and an over-prediction still refuses.
OpenStack's projection (miss -1.3%) regenerates identically. Wikimedia's sealed window projects to
at least 2,848 changes over its twelve months (captured 0.984 at the 2027-01 fetch), and its
simulation is read at that size.
The assumption the lower bound rests on, stated. An independent review of this change confirmed
that a smaller planned window moves every registered quantity of partition_sensitivity.py in
the conservative direction (the detectable effects grow, a null reads bounded less often,
reproducibility fails more often; sigma_run is calibrated on the pilot's changes, independent
of the planned size). What the projection cannot show is the other half: an under-prediction
caused by capture is bounded by the test window's capture (0.984, at most about 1.6%), but one
caused by arrivals is a lower bound only if the test window's arrivals stay at or above the
training rate. Monthly arrivals rise into the dev window without establishing that they stay
there. Registered check at fetch: when the sealed window is fetched, its change count is
compared with the projection, and a realised window below planned_changes is reported beside
the verdict with the simulation's power at the realised size. The projection artifact records
dev_check.reading (holds, lower_bound or refuse) in place of the earlier holds flag,
so the tolerance is tested once.
Review of check 5 and the projection (2026-10-01). An independent reviewer reproduced every
figure in the planted and projection entries and found nothing blocking, plus three guard gaps
in check 5, all fixed and tested: a planted run from another organization passed the check (it
now must be the read organization's two halves at the admissible list's training size), a run
planted in one half only passed (every half must carry the plant), and the check was optional
(partition_pilot.py now refuses a test-window read without --planted).