Skip to content

Measurement

At K = 24 the gate holds its level and two draws of splits agree: the fixed-K simulation

partition-sensitivity-openstack.json (commit 59f48bf; every input read from its artifact and recorded under inputs). 1,000 null studies and 300 per power step at each point, the 1,635 planned changes, K = 24, N = 1,850. At each point on the pilot's per-run spread, sigma_run is the shift at which 24 null runs on the pilot's 206 changes spread that much (100 studies a step):

OpenStack on corpus v3: seed effect 0.0077, detectable +0.0246 and +0.0210, and a dev pilot that passes on a different partition

Five single-seed placebo jobs at the half size (207303 to 207307, pinned at 8334d79) on corpus-v3/openstack, copied to TIGRIS with all 109 files' sha256 matched. The five roots are byte-identical under the same digest command as v2: 0ee3f74ab4a1b6e9. Every outcome-neutral check passed in every job. The halves equalize at 2,004 and are scored on 98 and 109 development changes (v2: 116 and 113).

OpenStack on corpus v2 at the half size: the seed effect holds five seeds, and the dev-window H1 pilot is inconclusive at +0.013

The first GPU runs on corpus v2. The v2 OpenStack corpus was copied to TIGRIS (corpus-v2/openstack, every file's sha256 matched) and the placebo run at the half size under RUN_TAG=v2, five single-seed jobs (206166, 206171 to 206174), each building its halves into its own root so concurrent builds could not overwrite one another. The five roots are byte-identical, checked on the cluster with find . -type f ! -name '*.source.json' ! -name placebo.json -print0 | sort -z | xargs -0 sha256sum | sha256sum in each root: every one gives f49b745cfbde62c0. --corpus-digest records that value when the runs are merged; it is the caller's assertion and the merge does not recompute it. merge otherwise refuses runs whose corpus sources differ, and with the digest compares them with only the root removed, mode, windows and every count still compared. The halves equalize at 2,158 examples, of which 2,155 trained (three refused by the trainer), and are scored on 116 and 113 development changes.

The human check: 77% raw agreement with rater A, below the model pair, and outside names is where the human and the raters part

Read under the rules registered above and on 2026-09-24, on the answers as locked (label-audit-v2.json, regenerated; the model raters' labels and all-item pairs reproduce unchanged). Every human pair a reported slip touches is also read without that item. The model pair is re-read on the 152 checked items, the comparator for every human row:

Registered: negative readings are bounded by each cell's detectable effect, and rebase edits get a sensitivity analysis

Negative readings. The proposal of 2026-09-23 is adopted. A cell reads bounded when its interval's upper bound lies below the effect the sensitivity analysis says that cell detects at the Holm level it is read at; the gate reads the bounds from decomposition-sensitivity.json (detectable_effects) and refuses a confirmatory cell without one at every level. The SESOI band is reported beside every cell and decides nothing. Justification: Lakens, Scheel and Isager (2018) list the effect a study was designed to detect among the bases for an equivalence bound, and Lakens (2022) asks a design to say which effects it is informative about when it cannot resolve the SESOI. The bound is reachable where absence was not: the regenerated sensitivity artifact records null_reads_bounded per cell and level beside null_reads_absent: a true null reads bounded 0.97 of the time in three of the four H1 cells and 0.94 in Qt's at 95%, against 0.000 absent in all four; every other figure in the artifact reproduced exactly. The bounds are recomputed at N on corpus v2 before the seal opens. A reversed effect now reads bounded, since H1 is directional.

FDLoRA's implementation, corrected before it ran: what job 183579 would have measured wrong

An independent review of PR #36 found the round-0 average (Algorithm 1, line 7) was computed per source rather than over every client the plan builds, before job 183579 reached its queue slot. The paper's N is every client the server sees; this repository's sources name separate simulated federations for the attack scripts, and the two are not the same population. Fixed by flattening plan into one list before Stage 1 runs and averaging over all of it, so theta_s0 is one value shared across every client regardless of source. Job 183579 was cancelled as invalid before it produced a result; nothing from it is quoted anywhere.

The audit's kappa is the prevalence paradox: AC1 0.853, and the raters part mostly on whether an item is valid

Rater A calls 311 of 383 items valid and rater B 323, so both raters' marginals sit on one label and chance agreement under kappa is high. That depresses kappa at high raw agreement (Feinstein and Cicchetti 1990), which is why kappa is reported beside raw agreement and a paradox-resistant coefficient rather than alone (James, LREC 2026, arXiv 2603.06865). Gwet's AC1 takes chance agreement over the whole five-label scale (Gwet 2008). Same items, same bootstrap seed, so the kappa intervals reproduce byte for byte (label-audit-v2.json, regenerated):

Label audit v2: two blind model raters agree at kappa 0.56, and validity does not differ between the units a contrast compares

The first audit's context-dependent class folded two questions together: whether the request can be understood from what the example shows, and whether the rewrite uses names the example does not show. A clear request answered with a project's own constant is a valid label that no prompt alone reproduces, which is house-style knowledge an adapter can learn, not label noise. Rubric version 2 asks the two separately (scripts/label_audit/rubric.md).

At the test window's size each H1 cell detects about three exact-match points, and absence is out of reach

scripts/decomposition_sensitivity.py simulates one H1 cell as the gate reads it: the placebo's own-against-sibling contrast on each half as the variance model, the test window's projected changes split between the halves in their dev-window proportion, five seeds at each organization's half-size seed effect, and the stratified crossed interval. Marginal power 0.928 per cell, so three independent cells pass together about four times in five. Generated from decomposition-sensitivity.json (100 trials a step, 1000 resamples).

The gate's stratified interval holds nominal at both Holm levels, in H2's regime

scripts/crossed_coverage.py --strata 2 simulates the interval the decomposition gate reads: two equally weighted strata, changes resampled within each, five seeds, 4,000 trials a cell (Monte Carlo error about 0.002 at the 0.0125 level). One-sided false-positive rate, the lower bound above zero under a true null, beside the unstratified crossed interval on the same runs. Generated from stratified-coverage-0.975.json and stratified-coverage-0.95.json.

Sibling-half leakage is below the registered threshold on the dev window

H2 would credit the organization with shared boilerplate if a sibling half's training data already held near-copies of the evaluated half's refinements. scripts/sibling_leakage.py assigns halves exactly as the placebo does and measures, for each half's dev window, the share of examples whose closest training example reaches a Jaccard threshold: from its own half's train window, its sibling's, and the foreign organization's. Generated from datasets/results/sibling-leakage.json.

The seed effect at the half size is above 0.01, so the decomposition runs at five seeds

The registered seed rule says three seeds up to a seed main effect of 0.01 and five above it. It was measured at 0.000 on the full organizations and 0.013 at about 1,800 examples; the re-registration trains on halves, so the rule has to be read at the half size. The three-seed placebo runs already are that measurement: OpenStack's halves trained on 2,157 and 2,159 examples, at the size every half-adapter will train on, and Qt's on 4,205 and 4,206. Read with scripts/seed_effect.py over each placebo's three single-seed runs.

Rank 256 at three seeds: capacity is not why the null arm is null

The conditional analysis pre-commits to rerunning both adapters at rank 256 when an arm returns no gain, on the argument that LoRA learns less than full fine-tuning and a null at rank 32 is equally consistent with insufficient capacity. That branch is now executed, at the registered three seeds. Read with scripts/reading.py --registered --markdown.