Skip to content

Measurement

Registered before computing: the placebo gate, and how it will be read

The registered gate compares an adapter trained on one organization against one trained on another, on the first organization's held-out refinements. It cannot say on its own that the organization is what the adapters learned. A project, a codebase family or a document type would all produce the same reading, and this study's own separability probe already reports that two projects inside one organization separate as well as two organizations do (0.828 and 0.872 within, against 0.849 across, on matched Python).

What FedDPA withholds is the half that identifies best

*-dual-cpp-rebuilt-c128-t2-local.json. The same run, the same clients, the same two rounds: the local adapter each client keeps, read by the two instruments that read the global one it sends. Both halves had two training passes over the same 128 examples, so this comparison carries none of the length confound that separates the dual run from the single-adapter one. Neither FedDPA nor FDLoRA makes it.

The adapter-instance cut transmits the source, and its own schedule sharpens it

*-dual-cpp-rebuilt-c128-t2.json. FedDPA's iterative variant on the rebuilt corpus, 128 examples a client, two rounds: each client holds a global adapter that is communicated and a local adapter that never is. What the server receives is the global half, and that is what the two instruments read here. Neither FedDPA nor FDLoRA attacks the half that leaves; FedDPA states the assumption instead, that "all clients are trusted ... and the whole process does not suffer from any attacks".

Cluster size carries no information about the contrast, measured

scripts/cluster_informativeness.py, datasets/results/cluster-informativeness.json. The registration binds the gate to the pooled estimand, and one of its two reasons is Kahan et al.'s condition: pooling over examples is wrong when a change's size carries information about its outcome. The report quoted a correlation for that, and nothing on disk produced it.

Nine sources read in full, and what reading them changed

# research(2026-09). The Stage 1 report carried ten \needcite badges because the intake staged every entry as a search-level characterization. All but one are now cited: each source was read against arXiv, the ACM Digital Library, the ACL Anthology or Springer, and four of the claims built on them were wrong in a way a reviewer would have caught.

The interval's own false-positive rate, measured rather than quoted

datasets/results/interval-calibration.json, scripts/interval_calibration.py. Section 7 of the report states the pairs cluster bootstrap's false-positive rate under a true null. That figure has been living in a comment in measure/stats.py since it was measured, with no artifact behind it, which is the one thing a registered report cannot do with a number a reviewer will check.

Repack the same clients and the detector survives; the classifier does not

*-c128-p2.json. The 128-example run said leakage is a function of how long each client trains. It was one packing, so who shared a client with whom was decided once, and the reading rested on two points that differ in size as well as in length. This is the same 51 clients, the same training seed, the same 128 examples each, repacked with packing seed 2: the only thing that moves is which examples group into which client.

The probe's intervals could not contain their own estimates, and now do

The review found the raw probe reading 0.841 with an interval of [0.814, 0.836]. The cause was the procedure, not the interval formula: each bootstrap resample refitted the classifier on changes drawn with replacement, about 63% of them unique, so every refit trained on less and scored lower, and the resampled accuracies sat below the estimate by more than their own spread. A bias-corrected (BCa) interval was tried first and made it worse -- [0.839, 0.839] -- because any interval built from quantiles of the draws cannot contain an estimate that every draw falls below.