RQ1 on the study's own windows¶

The first run using the train and dev windows rather than a random holdout from one month. Both organizations, base plus both adapters on both dev windows, equalized training, one seed. datasets/results/rq1-windows.json. Completed in 2:19:50 against a 4:30 request; all seven outcome-neutral checks pass and the apparatus holds.

Gate: fail, under both estimands. Neither lower bound clears zero, and OpenStack's misses by 0.003. Normalized exact match agrees throughout (+0.0230 and +0.0110 pooled), so this is not an artifact of the strict metric.

The first thing that is new. Both contrasts are positive. The two earlier pilots had opposite signs, and the signs swapped when training sets were equalized, which left run-to-run variation as large as the effect. Here both organizations lean the way RQ1 predicts, on windows chosen by the design rather than by a random split.

The second thing, which matters more. Adaptation is enormous and organization-specificity is tiny. Training moves exact match from 0.048 to 0.306 on OpenStack, roughly a sixfold gain of 26 points. The organization-specific part of that is 2 points. An adapter trained on OpenStack scores 0.308 on Qt's held-out refinements, against 0.319 for Qt's own adapter, and actually edges its own organization's 0.306. Whatever the adapters learn is almost entirely the task -- the form of a refinement, the conventions of a diff-shaped answer -- and almost none of it is the organization. If a fingerprint exists it is about one thirteenth the size of the adaptation effect it rides on.

What this does to the power argument, and it is not what scaling suggested. A square-root extrapolation from the pilot's +0.030 at 880 changes gives about +0.021 at the test window's projected size, which would have put the observed effect exactly at the detection threshold. Running the simulation on these variances instead (datasets/results/power-rq1-windows.txt, 400 trials, 1,000 resamples, tolerance 0.005) says otherwise:

Qt binds, and the earlier analysis said OpenStack did. That reversal is the finding. What binds is not the smaller MDE but the smaller ratio of effect to MDE, and Qt's observed effect sits below what its own sample can detect. The gate requires both organizations to clear zero, so on these estimates RQ1 fails on Qt even if the effect is real in both. OpenStack, by contrast, has about 50% headroom where the scaling argument predicted none.

Three things this does not license. The MDE and the observed effect come from the same run, so "observed exceeds MDE" is a statement about this sample's variance structure, not a guarantee about the test window. One seed underlies both. And the dev windows are censored unequally, which is the direction that would most distort Qt's estimate relative to OpenStack's.

The Stage 1 decision it forces is real, and belongs at registration rather than after: a pass rule requiring both organizations is, on current estimates, a rule Qt cannot satisfy. Either that conservatism is accepted and stated, or the rule is something else, and either way it is fixed before the test window is opened.

What this is not. One seed, not the registered three. The dev windows, not the sealed test window. And both dev windows are censored, OpenStack missing an estimated 37.0% of its example-bearing cohort and Qt 17.9%, unequally and against the slow reviews; Qt's window is also one month of two because of the ban, which is why it carries 175 changes against OpenStack's 235 and has the wider interval despite Qt being the larger corpus. This is the best available pre-registration estimate of RQ1. It is not the RQ1 answer.

Training, for the record. Both adapters ran 542 steps on equalized sets, 4,321 and 4,323 items after refusals, reaching final-epoch mean losses of 0.288 and 0.221 from first losses of 0.733 and 1.060. Adapter weight norms 18.41 and 18.26, close enough that neither adapter can be said to have moved further than the other.