Skip to content

Measurement

s split by token class: OpenStack's development contrast is not vocabulary alone, and it sits in the edit

# research(2026-10). Review panel item 5 asked whether s measures an organization's vocabulary rather than its conventions, since a sibling adapter has seen the organization's names and a foreign one has not. likelihood_score.py now classes each target token as identifier or other and as changed or copied, and stores each class's contribution to the mean per token (sphragis/experiment/tokens.py): identifiers are Pygments' names other than the language's builtins, a token touching one counting as one, so other is conservative; only the languages these corpora hold are lexed, with the suffixes Pygments misreads pinned, and anything else reads as text; changed or copied comes from aligning lines and then, within replaced lines, words and symbols with the reviewed hunk. The contributions add back to s within fp32 rounding: identifier, other and end, or changed, copied and end. This replaces the Stage 1 draft's plan to reread both hypotheses on identifier-renamed references: renaming moves every arm off the distribution it was trained on and removes naming conventions, which are house style too.

The bounded reading becomes a one-sided equivalence test at the SESOI

# research(2026-10). A cell read bounded when its upper bound fell below the effect its simulation says it detects at its Holm level: a margin set by the design's size, not by what matters. Equivalence bounds are set by the smallest effect size of interest (Lakens, Social Psychological and Personality Science 8(4), 2017; Lakens, Scheel and Isager, AMPPS 1(2), 2018, which also describes bounds at the smallest effect a study has the power to detect), and this study has a justified SESOI, δ (2026-09-30 and 2026-10-09 entries). So a cell now reads bounded when its upper bound at its Holm level lies below δ (decomposition.cell_verdict, margin): one-sided, since both hypotheses are directional, so a reversed effect still reads bounded. A cell's simulation stays required before a test read, as its power, not its margin; a cell without one is still never read bounded.

The gate reads H2's confirmatory cell on C++ examples against the matched foreign arm

decomposition.design(admitted, foreign_source, h2_withdrawn=...) implements item 1 of the plan above: the pair both ways when Qt and Chromium are admitted, otherwise one Qt cell against foreign_source, one of FOREIGN_SOURCES, refused without Qt admitted. No read takes the source from a caller: registered_design reads FOREIGN_SOURCE, set by the commit that fixes the source before the first H2 development contrast, and H2_WITHDRAWN, set if the foreign pool fails check 4, which leaves H1 alone at the whole family level. The gate, both supplements, registered_read and h1_test_gate all read it, so no source can be tried against a contrast and then chosen. Every confirmatory H2 cell reads its organization's C++ examples (H2_POPULATION, filtered lazily through Restricted, so rows of organizations H2 does not read are never parsed) against the arm foreign_arm(source, sibling) names, one per evaluated half (matched_clusters, through paired_clusters). decomposition_gate keeps H1's verdict on every example and takes the joint shares, the sum and H1 restricted to C++ on the C++ examples, labelling each cell and joint reading with its population; exploratory cells are unchanged. whole_hunk_supplement replaces cpp_supplement: the same matched contrast over every example, reported only. An H2 development read also needs H2 bounds, which only the H2 simulation supplies, so none can run yet. The matched sampler and the partition jobs that train the foreign arms are the next step; no H2 contrast has been run.

Planned before any adapter d_org on s is read: H2 as a Qt cell against a C++-matched foreign adapter

# research(2026-10). Decided in the research-question review of 2026-10-14 (AJ's full-auto grant; the host questions at the end stay his), and revised the same day after an independent review. The mock review panel ranked H2 resting on two pending permissions as the likeliest reason for a continuity acceptance (papers PR #85, org-house-style/REVIEW-PANEL.md, item 1), and the registered rule makes H2 exploratory with near certainty: Chromium's infra-dev list has sent only an automatic reply to the request of 2026-09-23, and its freeze by 2026-10-23 is also blocked on cutting chromium/src below the project and on sub-month fetching past the host's 10,000-result cap (Plan A).

Wikimedia at K = 20: detectable effects below the SESOI, the null within nominal at the sizing bound

# research(2026-10). partition-sensitivity-wikimedia-rp1.0-logprob_per_token-k20.json (TIGRIS 256458, pinned at 642f6d9, RUNS=20), the simulation the K coverage rule queued. Pool, noise and shrinkage as at K = 10; 1,000 trials and 4,000 null trials a point, 2,848 planned changes. Detectable effect at one to three cells, in nats per token:

The H1 simulation on likelihood: detectable effects below the SESOI, and a null that runs hot at K = 10

# research(2026-10). partition-sensitivity-<org>-rp1.0-logprob_per_token.json (TIGRIS 251070 and 251061, pinned at e6fc180), scripts/partition_sensitivity.py --metric logprob_per_token. The pool is each pilot's 22 runs averaged per example and shrunk toward the mean contrast by the share of its spread that is not noise (OpenStack 0.958, Wikimedia 0.701); each simulated run adds the change-by-run noise measured on its pilot (per example 0.1291 and 0.1320, shared by a change 0.0162 and 0.0249) and a run shift calibrated to the pilot's per-run spread. A first version that put all run-to-run spread in the run shift, and one that kept 1/22 of the noise in the truth, were reviewed and replaced before these runs. K = 10 on both, the floor the SESOI sizes; planned changes 2017 and 2848. Detectable effect at one to three cells, in nats per token:

H1 on likelihood: check 5 passes at half and a quarter, and OpenStack carries a contrast

# research(2026-10). Read under the rules fixed in "Planned before any likelihood result is read" above, committed (ea562bf) before the 76 likelihood jobs were pulled. likelihood-partition-*-rp1.0.json, likelihood-sesoi.json (74e8195, computed by rule 3 as fixed in ea562bf, which leaves no free parameter; the job logs committed with it print each run's arm means), partition-pilot-<org>-likelihood.json and -likelihood-total.json.

The rules-file comparator read: no rules file carries a half-split contrast

The 44 TIGRIS jobs pinned at 05465f1, every one completed (logs in datasets/logs/): 22 distillations, 229778 and 229779 the written guides and 229780 to 229799 the first ten admissible partitions of each organization (19.2 GH200-hours, 40 to 85 minutes a partition), and 22 ARMS=rules evaluations, 229800 to 229821, each queued behind the files it reads (12.0 GH200-hours). Read by retrieval_read.py --arms rules (rules-reading-openstack.json, rules-reading-wikimedia.json) with the retrieval foreign jobs as the base arm. The adapters' row reproduces the retrieval read's exactly, so both reads cover the same partitions and examples. Generated from the artifacts:

The retrieval comparator read: every own-minus-sibling reading inconclusive

The 22 TIGRIS jobs planned on 2026-10-05 (223007 to 223028, pinned at d077d54, every one completed; logs in datasets/logs/), read by retrieval_read.py as merged (retrieval-reading-openstack.json, retrieval-reading-wikimedia.json), on the development window over the first ten admissible partitions, 501 and 711 examples, none dropped. The adapters' row is their own minus sibling over the same partitions and examples (rank 32). Generated from the artifacts:

Power at a test window's realised size implemented

"Fetch horizon" registers that a test window holding fewer changes than its simulation projected is reported beside its verdict with the simulation's power at the realised size. scripts/power_at_size.py computes it. It takes the test report and the simulation the report was read under, and refuses a simulation whose organization, K, projected size, calibration or bounds differ from the report's, a report at or above its projection, and a corpus whose rebuild does not give the simulation's pilot changes and partitions. It reruns the simulation's trials, at its seed, at the number of changes the cell was read on, at the lift that registered each level's bound and at the calibrated run shift. Neither the lift nor the shift depends on the test size, so no calibration or bisection is redone. It records the share of trials supported at each level and its Monte Carlo standard error, sqrt(p(1 - p) / trials), as Morris, White and Crowther (Stat. Med. 2019) ask of simulated performance. Power is at the registered bound, never at an observed effect. Both gates take these artifacts as powers, require one for every organization in below_projection and none for any other, check each Monte Carlo SE against its power and trial count, and report each level's power in the cell's size. partition_sensitivity.py now records its seed, and the pool and partition rebuild both scripts use is one function (pools).

The replication gate implemented beside design()

replication() and replication_gate() in sphragis/experiment/decomposition.py read the GitHub family as registered on 2026-10-04. The family takes only Apache, LLVM, .NET and Grafana, in that order; a Gerrit organization is refused there, as design() refuses a GitHub one, and a registered organization outside the members is reported as not collected.

The readings across organizations implemented, before any test window

sphragis/experiment/across.py, and four fields on every H1 cell from h1_over_partitions: meaningful (lower bound above the SESOI, per level), p_one_sided (the share of the cell's own bootstrap draws at or below zero, the registered p-value), bootstrap_se (the draws' standard deviation, the cell's weight in the summary) and resamples (the draw count, which the reading checks against the level). Choices made in the implementation, each before any reading exists:

The registered 7B on a 40 GB A100: the registered rank fits only at the margin, and the rank-256 and dual-adapter runs do not

memory-probe-sporc.json, SPORC job 21796973 (A100-PCIE-40GB, 42.43 GB as the device reports it, pinned at 592b11e). scripts/memory_probe.py trains the registered 7B with train_adapter for two optimiser steps a cell on synthetic items of fixed length, at the registered rank and at 256, then runs the registered fp32 generator over a prompt one token under the training bound. Training runs one item per micro-batch, so the longest item sets the peak, and no item over 2,048 tokens can be trained on. The CUDA context and libraries hold 0.55 GB outside PyTorch's allocator.

Target power is 0.95 for every hypothesis test, and the simulation reruns at it

Audit item 8. 0.928 was 0.80 joint over three independent cells, a target of our own making that no longer describes the admitted set. Replaced by the registered-report standard: "For frequentist analysis plans, the a priori power must be 0.95 or higher for all proposed hypothesis tests" (Nature Communications / Nature Human Behaviour Registered Reports author guidelines, read from the guidelines PDF). The hypothesis tests are H1 and H2, not their cells. H1 passes only when every admitted organization's cell is supported (intersection-union), and the organizations are independent, so H1's power is the product of its cells' powers: with k cells each needs 0.95^(1/k), 0.9747 at two and 0.9830 at three. The admitted set is fixed at Stage 1 submission, so the simulation finds the bound for each of one to three cells, and the design reads the one matching the set it registers.