Skip to content

October 2026

Wikimedia's development-window pilot: K = 16, and H1 inconclusive at +0.0009

Twenty-two partition_run jobs on corpus-v3/wikimedia, the first 22 admissible partitions at training seeds 1 to 22 and N = 1,850 (jobs 210577 to 210600, pinned at ec8cc05). Every outcome-neutral check passed and the apparatus held in all 22; the largest leakage rate, own or sibling, is 0.40% against the registered 2%. Every run scored the same 711 examples (429 changes); none was dropped.

The planted check at a half passes 10 of 10 runs: registered at a half, read on the planted run only

The ten runs planned above, repeated at PLANT=0.5 (jobs 210234 to 210243, pinned at b05a2d8), rq1-partition-openstack-*-n1850-plant0.5.json. All ten pass. Every half's lower bound is above zero (least +0.017); per-half estimates run from +0.053 to +0.142. The pass probability projected by the logged rule (normal per half, interval half-width over 1.96 as the standard error, product over halves) is at least 0.9989 for every run and 0.9999 on average. Both conditions of the rule hold, so check 5 is registered with the plant at a half: neutral.planted_convention, which reads the plant's strength and window from the run's corpus path and refuses any other, and partition_pilot.py --planted, which halts the cell when it fails.

The planted check at a quarter passes 2 of 10 runs: redesigned before registration

The ten runs planned above (jobs 210077 to 210086, pinned at b05a2d8), rq1-partition-openstack-*-n1850-plant0.25.json. Two pass (partitions 2 and 12), seven read mixed, one fail. The planted effect is there in every run but one half (partition 5, half b, -0.013), from about +0.003 to +0.106, and the development window's 81 to 125 changes a half resolve the smaller ones no better than the pilot's 183 did. Registered as drafted, check 5 would halt a working apparatus in about four studies of five. By the rule fixed before these runs, it is not registered at a quarter: the plant is raised to a half and the same ten partitions and seeds are repeated.

Planned before any run: the planted-convention check's pass rate on v3 at N = 1,850

Audit item 5. The Stage 1 draft registers outcome-neutral check 5: on the development window, one run with each half's own comment marker planted in a quarter of its refinements must pass on both halves. Its one pilot (v1, bf16, random halves of 183 changes each, one seed) passed on the weaker half at a lower bound of +0.005, and on a partition the halves hold 125 and 81 development changes. A positive control that halts a working apparatus is not "sufficient" in the sense the registered-report guidelines use, so its pass rate is measured before it is registered.

Planned before any run: a local decision model as a third blind label-audit rater

Exploratory, outside the registered pipeline. Ollama 0.35 serves decision models over a System One API (/v1/systemone): a choice question returns a probability per option and a yes/no question the probability of yes, in one pass, with no generated text. Nimble (Bespoke Labs, 9B, Qwen3.5-9B base, Apache 2.0) runs locally, so no corpus text leaves the machine; TypeSafe's hosted Jev, the API it follows, is not used, since its data terms are unpublished. The only published evaluation outside its vendors found a System One model's calibration model-specific, to be audited per model (Rafe and Das, arXiv:2609.24052).

Target power is 0.95 for every hypothesis test, and the simulation reruns at it

Audit item 8. 0.928 was 0.80 joint over three independent cells, a target of our own making that no longer describes the admitted set. Replaced by the registered-report standard: "For frequentist analysis plans, the a priori power must be 0.95 or higher for all proposed hypothesis tests" (Nature Communications / Nature Human Behaviour Registered Reports author guidelines, read from the guidelines PDF). The hypothesis tests are H1 and H2, not their cells. H1 passes only when every admitted organization's cell is supported (intersection-union), and the organizations are independent, so H1's power is the product of its cells' powers: with k cells each needs 0.95^(1/k), 0.9747 at two and 0.9830 at three. The admitted set is fixed at Stage 1 submission, so the simulation finds the bound for each of one to three cells, and the design reads the one matching the set it registers.

The test window re-projected over its twelve months on v3: 2,018 OpenStack changes

project-windows-openstack-v3.json, from window-report-openstack-v3.json (both new; the projection now writes an artifact and reads the corpus it is given). v3's train window holds 1,645 changes over ten months, captured at 0.952, so 172.9 true changes a month; that rate predicts the dev window at 211 against an actual 214 (-1.3%, inside the 15% tolerance). Over the registered twelve months, fetched at the earliest month the horizon permits (2027-01, captured 0.973), the test window projects to 2,018 changes, replacing the ten-month 1,809 scaled to v3 (1,635). The simulation now reads its test size from this artifact (--projection), not a typed --size.

An audit of this log against its artifacts: nine conclusions in force are wrong or unsupported

A reader with no part in writing these entries went through the log oldest to newest, re-deriving figures from the committed artifacts, and checked the current docs and the Stage 1 draft against it. Two findings were re-checked by hand before anything was acted on (the leakage halt and the test size, below). What stands, by consequence:

At K = 24 the gate holds its level and two draws of splits agree: the fixed-K simulation

partition-sensitivity-openstack.json (commit 59f48bf; every input read from its artifact and recorded under inputs). 1,000 null studies and 300 per power step at each point, the 1,635 planned changes, K = 24, N = 1,850. At each point on the pilot's per-run spread, sigma_run is the shift at which 24 null runs on the pilot's 206 changes spread that much (100 studies a step):