Skip to content

Research log

Pooling does not make absence reachable either

The registered pooled estimate is sharper than any one cell, so it was the natural home for the negative readings. Simulated the same way, H1 pooled over openstack, qt with every half one equally weighted stratum (planned changes [814, 995, 1784, 1497]): a true null reads absent 0.040 of the time at 97.5% and 0.145 at 95% (decomposition-sensitivity-pooled.json). Chromium would add a third organization and narrow the interval by roughly a further fifth, which does not change the picture. At this test window, 0.01 is not a resolvable equivalence bound.

At the test window's size each H1 cell detects about three exact-match points, and absence is out of reach

scripts/decomposition_sensitivity.py simulates one H1 cell as the gate reads it: the placebo's own-against-sibling contrast on each half as the variance model, the test window's projected changes split between the halves in their dev-window proportion, five seeds at each organization's half-size seed effect, and the stratified crossed interval. Marginal power 0.928 per cell, so three independent cells pass together about four times in five. Generated from decomposition-sensitivity.json (100 trials a step, 1000 resamples).

The gate's stratified interval holds nominal at both Holm levels, in H2's regime

scripts/crossed_coverage.py --strata 2 simulates the interval the decomposition gate reads: two equally weighted strata, changes resampled within each, five seeds, 4,000 trials a cell (Monte Carlo error about 0.002 at the 0.0125 level). One-sided false-positive rate, the lower bound above zero under a true null, beside the unstratified crossed interval on the same runs. Generated from stratified-coverage-0.975.json and stratified-coverage-0.95.json.

Sibling-half leakage is below the registered threshold on the dev window

H2 would credit the organization with shared boilerplate if a sibling half's training data already held near-copies of the evaluated half's refinements. scripts/sibling_leakage.py assigns halves exactly as the placebo does and measures, for each half's dev window, the share of examples whose closest training example reaches a Jaccard threshold: from its own half's train window, its sibling's, and the foreign organization's. Generated from datasets/results/sibling-leakage.json.

The seed effect at the half size is above 0.01, so the decomposition runs at five seeds

The registered seed rule says three seeds up to a seed main effect of 0.01 and five above it. It was measured at 0.000 on the full organizations and 0.013 at about 1,800 examples; the re-registration trains on halves, so the rule has to be read at the half size. The three-seed placebo runs already are that measurement: OpenStack's halves trained on 2,157 and 2,159 examples, at the size every half-adapter will train on, and Qt's on 4,205 and 4,206. Read with scripts/seed_effect.py over each placebo's three single-seed runs.

Chromium scoped: five C++ projects clear the rule, and no split of them meets the criteria

datasets/results/host-scoping-chromium.json, written by scripts/host_scoping.py from the raw counts committed beside it; about 1,050 requests at one a second, nothing written into the corpus. The selection rule (host-scoping-chromium/selection-rule.txt) was written at 21:58 EDT, before any per-project yield was read: C++ the dominant file type among what sampled human changes touch, and at least 256 projected examples over the train and dev months (2024-11 to 2025-10, cutoff 2024-11-01). Projected examples are human changes times reviewer-anchored comments per sampled change times 0.42, which is Qt's 0.219 examples per fetched change over those months (its examples and fetch records on disk) against its 0.5 comments a change of 2026-09-21. The 0.5 comes from ten changes, so the projections give orders of magnitude and nothing finer.

Rank 256 at three seeds: capacity is not why the null arm is null

The conditional analysis pre-commits to rerunning both adapters at rank 256 when an arm returns no gain, on the argument that LoRA learns less than full fine-tuning and a null at rank 32 is equally consistent with insufficient capacity. That branch is now executed, at the registered three seeds. Read with scripts/reading.py --registered --markdown.

AOSP stopped reviewing in public, and it takes the language-matched arm with it

# research(2026-09). The gate compares an OpenStack adapter against a Qt one. The two organizations barely share a file type, so an adapter that learned nothing but the language would beat the other's on its own held-out data and pass. The window reports now carry the histogram the variables table promised: OpenStack's training window is 2,054 Python and 1,184 reStructuredText of 4,327; Qt's is 4,211 C++, 1,427 qdoc and 807 headers of 8,442. In the dev window Qt holds eight Python examples against OpenStack's 243, so the matched arm cannot be evaluated on this pair at all.

Registered before computing: the placebo gate, and how it will be read

The registered gate compares an adapter trained on one organization against one trained on another, on the first organization's held-out refinements. It cannot say on its own that the organization is what the adapters learned. A project, a codebase family or a document type would all produce the same reading, and this study's own separability probe already reports that two projects inside one organization separate as well as two organizations do (0.828 and 0.872 within, against 0.849 across, on matched Python).