Registered before computing: organizational coherence across training lengths¶

If an organization is detectable exactly when its projects resemble each other more than they resemble an outsider's, that property is one number per organization:

coherence(O) = mean cosine between clients of O on DIFFERENT projects
             - mean cosine between clients of O and clients of any other organization

Same-project pairs are excluded from the first term on purpose, since a project effect is not an organization effect. At 64 examples a client, within C++, AOSP's is positive and Qt's is not (Qt's cross-project pairs sit at 0.2523 against Qt-AOSP pairs at 0.2589).

Prediction, written before the 128-example vectors are read: AOSP positive at both lengths; Qt negative at 64 and positive at 128, because Qt's project-level permutation moved from p = 0.238 to the 1/84 floor between those lengths and coherence is the quantity that permutation should be tracking. If Qt stays negative at 128, then coherence is not what the permutation reads and the explanation above is incomplete a third time.

Outcome: the prediction failed on the part that mattered. Within C++, with 95% intervals from resampling each organization's projects:

AOSP is positive at both lengths, as predicted, and Qt moved toward zero, as predicted. But Qt did not become positive: at 128 examples it sits at -0.0005 with an interval centred on zero, while its project-level permutation reached the 1/84 floor at that same length. By the criterion written above, pairwise coherence is not what the permutation reads, and the explanation is incomplete a third time. (Qt's interval at 64 also covers zero, so "Qt is incoherent" was already stronger than the data at that length; "Qt is not detectably coherent" is what it supports.)