Corrections from an adversarial review of today's code¶

Three independent reviews reran today's analyses against the committed data. Findings that change what this log says, stated here before any is fixed:

1. The subspace-split headline was overstated, and part of it was wrong. "0.791 against 0.419 on the whole update" is withdrawn as stated. - The whole-update 0.419 is a broken baseline, below chance: the project-level null has mean 0.509, and 25 of 30 Qt clients are predicted as AOSP. The rise to 0.791 is against a class-bias failure, not a fair reference, so "projecting nearly doubles the attacker" is withdrawn. - Rank 4 was chosen after looking. Under the exact project-level null (84 relabelings), rank 4's shared part is p = 0.036, rank 8's 0.024, but the maximum over the reported ranks is p = 0.071. - Where the signal sits depends on k. At rank 1 the residual (the part a split keeps local) is the strongest cell, 0.837 at the 1/84 floor; at ranks 4 and 8 it is the shared part; at rank 16 neither is significant (0.119). And rank 1's shared row is degenerate, since every rank-1 projected cosine is exactly 1.0. - So "the fingerprint is in the part that gets shared, not the part kept back" is not established. The defence table's SDFLoRA row changes from "leaks, more than the whole update" to "depends on the rank of the shared subspace; not established either way". Two of the three cuts, not three, are measured as leaking, pending the review of the factor geometry.

2. Qt's permutation p is seed-dependent at the draws the test uses. Rerun over four seeds:

AOSP stays at 1/84 on every seed and geometry. The direction of every Qt comparison survives every seed tried: 128 examples below 0.04, 64 examples above 0.16, A-only below 0.03, B-only above 0.16. But the repeated phrase "Qt reaches the 1/84 floor" was true for seed 0 only and is replaced by the ranges above. The composition-matched test earlier today used seed 0 throughout; its conclusion (length, not repacking) still holds against these ranges, since no composition-matched draw fell below 0.071.

3. A stale figure. The entry "The organization is there, at the operating point an attacker actually has" still quotes Qt at p = 0.071. The corrected value is 0.238 at seed 0, and 0.17 to 0.33 across seeds.

Still open from the same review, and not yet fixed: the permutation's tests are text searches that every mutant tried survives; the enumeration counts arrangements rather than distinct groupings; the overwrite guard checks at job start while scripts write hours later, so two concurrent jobs with one output can still replace each other; three further outputs are unguarded or misnamed.

The factor-geometry review: the A-only result survives, and two statements about it change. The third review attacked "A alone identifies the organization better than B or the product" with every mechanical explanation it could construct, and none held: - label-shuffled and project-level nulls sit at chance, with the truth ranked first of 84; - norms are not the cue: a Gram matrix built from A's norms with a common cosine scores below chance (0.46 and 0.41), and cosines alone reproduce the result (0.791 and 0.733); - the large shared component suppresses the signal rather than creating it: removing the global mean raises A's AUC to 0.842 and 0.866; - equal-weighting the modules keeps A ahead (0.794 and 0.708 against B's 0.735 and 0.618), and A matches or beats B for AOSP in every module kind; - leaving each project out in turn, A beats B for both organizations in all nine runs; - training order does not explain it (Mantel p 0.26 on A, 0.46 on B), and each client's LoRA state is restored and checked, with a fresh optimizer and reseeded generator, before it trains.

Two things change in how it is stated. 1. The product and B are one geometry, not two. Their off-diagonal cosines correlate at 0.9998 (largest difference 0.006), because A barely moves from its near-orthogonal initialization, so B A is close to B A0. "A beats B and the product" is one comparison. 2. The claim belongs to the membership detector. FedAttr's paired-difference statistic runs backwards for Qt on both factors (its own clients -0.061 against a stranger's +0.014 on A), and on the A geometry the q and k projections alone put Qt below chance. The result is stated for the detector that scores a round's aggregate against a reference direction, and not for every statistic in the study.

(With content held to C++ there are two organizations, so the AOSP and Qt permutations enumerate the same groupings as complements. They are two statistics over one null, not two independent tests.)