s split by token class: OpenStack's development contrast is not vocabulary alone, and it sits in the edit¶
# research(2026-10). Review panel item 5 asked whether s measures an organization's vocabulary
rather than its conventions, since a sibling adapter has seen the organization's names and a
foreign one has not. likelihood_score.py now classes each target token as identifier or other
and as changed or copied, and stores each class's contribution to the mean per token
(sphragis/experiment/tokens.py): identifiers are Pygments' names other than the language's
builtins, a token touching one counting as one, so other is conservative; only the languages
these corpora hold are lexed, with the suffixes Pygments misreads pinned, and anything else reads
as text; changed or copied comes from aligning lines and then, within replaced lines, words and
symbols with the reviewed hunk. The contributions add back to s within fp32 rounding:
identifier, other and end, or changed, copied and end. This replaces the Stage 1 draft's plan to
reread both hypotheses on identifier-renamed references: renaming moves every arm off the
distribution it was trained on and removes naming conventions, which are house style too.
All 44 development runs the likelihood pilots read were rescored (PR #136, jobs pinned at
0c4f764). Of their 79,992 rows, 79,989 reproduce their stored log-probability exactly and three,
in one Wikimedia job, differ by at most 2e-6 nats, fp32 rounding; the whole-s pilots reproduce
partition-pilot-<org>-likelihood.json, OpenStack exactly and Wikimedia within 1e-11.
Development H1 own minus sibling, all 22 runs per organization, at the pilot's settings
(partition-pilot-<org>-likelihood-split-<component>.json, 95% crossed intervals; whole s:
OpenStack +0.0231 [+0.0097, +0.0376], Wikimedia +0.0058 [+0.0003, +0.0117]):
| Part of s | OpenStack | Wikimedia |
|---|---|---|
| identifier | +0.0122 [+0.0040, +0.0224] | +0.0011 [-0.0020, +0.0043] |
| other (not a name) | +0.0100 [+0.0013, +0.0203] | +0.0045 [-0.0001, +0.0095] |
| end token | +0.0008 [-0.0002, +0.0020] | +0.0001 [-0.0010, +0.0011] |
| changed by the revision | +0.0213 [+0.0094, +0.0352] | +0.0063 [+0.0017, +0.0113] |
| copied from the reviewed hunk | +0.0009 [-0.0023, +0.0037] | -0.0007 [-0.0030, +0.0015] |
| other and changed | +0.0093 [+0.0017, +0.0192] | +0.0047 [+0.0007, +0.0090] |
On OpenStack the part outside names carries 43% of the contrast, and its interval excludes zero at 95%; so do Wikimedia's changed part and its other-and-changed part. Almost all of each organization's contrast is in tokens the revision changed; the copied code carries none. Exploratory and development-window only: it decides nothing, and the test window will be read the same way as a registered secondary reading. Rows a lexer may have misread (identifier class changing when each line is lexed alone, or error characters) were dropped from every arm and the reading repeated: no estimate changed sign and no interval's verdict moved.