Skip to content

Apparatus

fp32 still reproduces across nodes without the penalty

# research(2026-10). scripts/determinism_check.py, datasets/results/determinism-sym-0-gh-a-001-rp1.0.json and determinism-sym-0-gh-a-082-rp1.0.json, the base model over the same 150 prompts at penalty 1.0 in two jobs on two nodes (TIGRIS 239444 and 243769, the second excluding the first's node rather than waiting on the node the 1.1 measurement used). fp32 differs on 0 of 150 predictions between the jobs; bf16 differs on 3 to 5 between them and on at most 1 within one process. The fp32 registration holds under the registered decoder.

Every stored partition run rescored at penalty 1.0: H1 stays inconclusive and K falls

# research(2026-10). The reruns the decoder entry above called for. Every development partition run and its planted variants on both organizations (76 runs, rq1-partition-*-rp1.0.json) and both placebo runs the H1 simulation reads (rq1-placebo-*-rp1.0.json) were rescored with their adapters reused (decoder_check.sbatch, pinned at ddb3d1b and 121d0c6). Partition 2's and partition 3's rescorings, run a second time in new jobs, kept every prediction, so rescoring at 1.0 reproduces across jobs in fp32. The retrieval and rules comparators were run again at 1.0 (44 jobs, *-rp1.0.json, pinned at 3903b90) and read with retrieval_read.py, which takes the -rp1.0 adapter runs. Own minus sibling throughout, the pilots' at 97.5%:

The checkpoint's repetition penalty, measured: plain greedy is registered

# research(2026-10). scripts/decoder_read.py, datasets/results/decoder-penalty.json. The three rescored partition runs ran at 3264e08, the pre-rebase copy of 09e1511, through the first --reuse-adapters, which records no test 3. Qwen2.5-Coder-7B-Instruct's pinned generation_config.json sets repetition_penalty: 1.1, and transformers 5.17.0 adds its processor whether or not it samples, over the prompt's tokens as well as the output's. Every result before this entry therefore decoded greedy at 1.1, while the registration says greedy. A code refinement is mostly a copy of the code under review, and a penalty on prompt tokens penalises the copy. Measured on stored adapters, rescored with only the decoder changed (decoder_check.sbatch, decoding_check.sbatch, fp32):

H1's AI-trailer and backport sensitivities implemented, read on the development pilots

Two of the readings registered beside the pass rule (registered decisions, 2026-10-02) recompute an H1 cell without a set of examples. scripts/sensitivity_ids.py lists them per organization and window from the corpus the partitions were built from (corpus-v3, deduplicated and split as the study loads it): examples whose (project, Change-Id) ai_trailers.py flagged, and examples whose (project, Change-Id) sits only on backport branches (sphragis.corpus.backports, now the one rule backport_share.py reads too). partition_pilot.py --without NAME=FILE reads the cell again without them, on the same runs, levels, seed and draws, beside the cell and binding nothing, and both gates carry the readings. On the development pilots (partition-pilot-<org>-sensitivities.json, 22 runs each; the registered cell reproduces the committed pilots exactly), generated from the artifacts:

Planned before any run: the rules-file comparator

The registered rules-file comparator (registered decisions, "Comparators and audits"), on the retrieval comparator's runner, a rules file in each pool's place. Design of record: docs/superpowers/specs/2026-10-05-rules-file-comparator-design.md. Exploratory, binding nothing.

Planned before any run: the retrieval comparator on the registered runner

The registered comparator (registered decisions, "Comparators and audits"), rebuilt so its contrasts are the adapters' contrasts with a pool where each adapter was. A first draft (#77, 2026-10-02) was amended before any generation, after an independent review found three biases. Its pools were whole halves of unequal size, with a foreign pool twice their size, and closer neighbours in a larger pool pull sibling minus foreign toward zero. It deduplicated each organization once, where the runner deduplicates each half. And it pooled the halves in one bootstrap. None of these survives here.

OpenStack and Wikimedia development pilots rerun to name their organization

partition-pilot-openstack.json and partition-pilot-wikimedia.json were rerun by the current partition_pilot.py on the same 22 runs each, in admissible order. A test read sizes K only on a pilot that names its organization (pilot_sizing), and the earlier artifacts did not. The cell, its intervals, its per-run values and the sizing are unchanged, so K stays 24 and 16, and each pilot's per-run spread still matches the calibration its simulation records (same_calibration), as a test read checks. The reruns add the fields the current pilot writes (org, window, p_one_sided, bootstrap_se, meaningful) and record the absent bounds per level.

H1's verdict over the admitted organizations' test reads implemented

h1_test_gate in sphragis/experiment/decomposition.py reads H1 the way replication_gate reads the GitHub family. It takes the organizations from ADMITTED_ORGANIZATIONS, never from its caller, and refuses to run before they are frozen. It also refuses while H2 is confirmatory (Qt and Chromium both admitted), because Holm cannot be read without H2 and H2's test read over partitions is not built. It takes one partition_pilot.py test report and one simulation artifact per admitted organization, exactly those. Each report is checked by require_readable at every registered Holm level and by require_test_read against registered_read, and each bound is read from the organization's own simulation (_registered_bounds, shared with replication_gate). H1 is the intersection-union over the cells at the registered levels through the existing holm_steps, with below_sesoi and the pre-committed reading. With OpenStack and Wikimedia admitted, H1 is read at 95% on two cells.

The 2026-10-04 pass and a sweep of September and October 2026 are now positioned in the papers intake (papers #54), with a watch list of queries per topic for the weekly sweep. Nothing found pre-empts the decomposition (own, sibling and foreign over repeated partitions) or the GitHub replication family. Two RQ2 claims are narrowed (ROADMAP, Plan F). An anonymous ICLR 2027 submission identifies a released LoRA adapter's training document from its weights, so RQ2's claim is source at the organization unit from the transmitted half of a split, not that adapter weights reveal their source in general. A second shows a post-hoc privacy audit can measure the shift between sources instead of leakage; RQ2 therefore attributes between language-matched organizations and reports the same attack between two disjoint draws from one organization beside each result. Owner-approved sharing of agent memory exists in deployment, so the owner-declared boundary is worded as declared per example, by provenance, before training. The Stage 1 report is RQ1 only and states none of these claims. Anonymous submissions are recorded by OpenReview id and cited only once public.

The open-weight rater is adopted by its rule, and it shows the two model raters agree with each other more than with anyone else

label-audit-v2-open-rater.json (labels label-audit-v2-open-rater-labels.json, SPORC job 21795722, six minutes on one A100). Qwen3-14B at pinned weights labelled all 358 items still in the refined corpus, none unparsed (the 25 audited examples a later refinement removed are left out, 13 of them human-checked, so the human pairs read on 139 items). By the rule fixed beforehand it is adopted: its label kappa with the human, 0.254 [0.102, 0.404], reaches the lower end of the locked human~A interval, 0.154. That bar was lenient by design, so the comparators on the same 139 items carry the reading: the human agrees with rater A at 0.289 and with rater B at 0.343, so the open rater sits just below A, its interval overlapping both. Collapsed to valid or not, its kappa with the human is 0.367.

Apache's own conversion, 0.281 [0.259, 0.304]; all six GitHub candidates now sized on their own pilots

github-conversion-apache.json. Apache's 2024-11 lists 11,083 PRs, all counted: 2,051 opened by a bot or agent were not collected, 2,241 carried a reviewer's inline comment, 3,370 examples were built and 2,453 survived refine. Against 8,736 reviewer threads the conversion is 0.281, 95% interval [0.259, 0.304], the highest and the tightest of the six; all 15 of the sizing's draws in the month recount the same. Its suggestion threads (1,851) are a small share beside .NET's, Grafana's and HashiCorp's. Across the six, a larger share of suggestion threads goes with a lower conversion, roughly (HashiCorp's is the largest share and the lowest rate, Apache's and LLVM's the smallest shares and the highest rates; .NET and OpenJDK break the order), as expected when a reviewer who writes the fix leaves an example refine removes.

HashiCorp's own conversion: 0.222 [0.169, 0.279]; HashiCorp stays borderline for both

github-conversion-hashicorp.json. HashiCorp's 2024-11 lists 2,194 PRs, all counted: 431 opened by a bot or agent were not collected, 282 carried a reviewer's inline comment, 444 examples were built and 131 survived refine. It is the only pilot where reviewers open more threads with a suggestion block (669) than without one (590), so most of its review edits are written by the reviewer, and refine removes them. Against the 590 the conversion is 0.222, 95% interval [0.169, 0.279], the widest of the five for having the fewest threads; all 17 of the sizing's draws in the month recount the same. On its own pilot HashiCorp projects 154 to 4,220 examples and stays borderline for an H1 cell and for RQ2. Its interval widens the span Apache borrows (Apache's census is running), which lowers Apache's low end to 8,720, still past an H1 cell (github-sizing-report.json). Five pilots in, the measured conversions run from 0.222 to 0.255.

Grafana's own conversion: 0.231 [0.202, 0.261]; Grafana reaches RQ2 again, narrowly

github-conversion-grafana.json. Grafana's 2024-11 lists 3,227 PRs, all counted: 973 opened by a bot or agent were not collected, 579 carried a reviewer's inline comment, 663 examples were built and 306 survived refine. Suggestion threads are 761 beside 1,325 other reviewer threads, as high a share as .NET's. Against those 1,325 threads the conversion is 0.231, 95% interval [0.202, 0.261]; all 15 of the sizing's draws in the month recount the same. On its own pilot Grafana projects 1,880 to 12,104 examples, so its low end clears RQ2's 1,850 by 30 and it is an RQ2 candidate again (it had dropped to borderline on the span borrowed from OpenJDK); H1 stays borderline. The four measured conversions so far (0.231 to 0.255) are close to one another and to the Gerrit range they replaced (github-sizing-report.json). Apache's census is running and HashiCorp's pilot month is being collected.