Skip to content

October 2026

fp32 still reproduces across nodes without the penalty

# research(2026-10). scripts/determinism_check.py, datasets/results/determinism-sym-0-gh-a-001-rp1.0.json and determinism-sym-0-gh-a-082-rp1.0.json, the base model over the same 150 prompts at penalty 1.0 in two jobs on two nodes (TIGRIS 239444 and 243769, the second excluding the first's node rather than waiting on the node the 1.1 measurement used). fp32 differs on 0 of 150 predictions between the jobs; bf16 differs on 3 to 5 between them and on at most 1 within one process. The fp32 registration holds under the registered decoder.

Every stored partition run rescored at penalty 1.0: H1 stays inconclusive and K falls

# research(2026-10). The reruns the decoder entry above called for. Every development partition run and its planted variants on both organizations (76 runs, rq1-partition-*-rp1.0.json) and both placebo runs the H1 simulation reads (rq1-placebo-*-rp1.0.json) were rescored with their adapters reused (decoder_check.sbatch, pinned at ddb3d1b and 121d0c6). Partition 2's and partition 3's rescorings, run a second time in new jobs, kept every prediction, so rescoring at 1.0 reproduces across jobs in fp32. The retrieval and rules comparators were run again at 1.0 (44 jobs, *-rp1.0.json, pinned at 3903b90) and read with retrieval_read.py, which takes the -rp1.0 adapter runs. Own minus sibling throughout, the pilots' at 97.5%:

The checkpoint's repetition penalty, measured: plain greedy is registered

# research(2026-10). scripts/decoder_read.py, datasets/results/decoder-penalty.json. The three rescored partition runs ran at 3264e08, the pre-rebase copy of 09e1511, through the first --reuse-adapters, which records no test 3. Qwen2.5-Coder-7B-Instruct's pinned generation_config.json sets repetition_penalty: 1.1, and transformers 5.17.0 adds its processor whether or not it samples, over the prompt's tokens as well as the output's. Every result before this entry therefore decoded greedy at 1.1, while the registration says greedy. A code refinement is mostly a copy of the code under review, and a penalty on prompt tokens penalises the copy. Measured on stored adapters, rescored with only the decoder changed (decoder_check.sbatch, decoding_check.sbatch, fp32):

  • in Corpus
  • 3 min read

The contamination battery read against blind baselines: the windows separate without the model

# research(2026-10). scripts/contamination_blind.py, datasets/results/contamination-openstack-6mo-blind.json. The battery compares a pre-cutoff window with a post-cutoff one, and the windows differ in time as well as in exposure. Duan et al. (COLM 2024, arXiv:2402.07841) attribute membership attacks' apparent success on such splits to "a distribution shift, such as when members and non-members are drawn from the seemingly identical domain but with different temporal ranges", and Das, Zhang and Tramèr (IEEE SPW 2025, arXiv:2406.16201, read from the paper) show "blind attacks -- that distinguish the member and non-member distributions without looking at any trained model -- outperform state-of-the-art MI attacks", with a bag of words and date detection. The Stage 1 novelty review (papers#64) flagged that the battery reads a near-zero gap as inconclusive but has no rule for a gap away from zero.

The rules-file comparator read: no rules file carries a half-split contrast

The 44 TIGRIS jobs pinned at 05465f1, every one completed (logs in datasets/logs/): 22 distillations, 229778 and 229779 the written guides and 229780 to 229799 the first ten admissible partitions of each organization (19.2 GH200-hours, 40 to 85 minutes a partition), and 22 ARMS=rules evaluations, 229800 to 229821, each queued behind the files it reads (12.0 GH200-hours). Read by retrieval_read.py --arms rules (rules-reading-openstack.json, rules-reading-wikimedia.json) with the retrieval foreign jobs as the base arm. The adapters' row reproduces the retrieval read's exactly, so both reads cover the same partitions and examples. Generated from the artifacts:

H1's AI-trailer and backport sensitivities implemented, read on the development pilots

Two of the readings registered beside the pass rule (registered decisions, 2026-10-02) recompute an H1 cell without a set of examples. scripts/sensitivity_ids.py lists them per organization and window from the corpus the partitions were built from (corpus-v3, deduplicated and split as the study loads it): examples whose (project, Change-Id) ai_trailers.py flagged, and examples whose (project, Change-Id) sits only on backport branches (sphragis.corpus.backports, now the one rule backport_share.py reads too). partition_pilot.py --without NAME=FILE reads the cell again without them, on the same runs, levels, seed and draws, beside the cell and binding nothing, and both gates carry the readings. On the development pilots (partition-pilot-<org>-sensitivities.json, 22 runs each; the registered cell reproduces the committed pilots exactly), generated from the artifacts:

The retrieval comparator read: every own-minus-sibling reading inconclusive

The 22 TIGRIS jobs planned on 2026-10-05 (223007 to 223028, pinned at d077d54, every one completed; logs in datasets/logs/), read by retrieval_read.py as merged (retrieval-reading-openstack.json, retrieval-reading-wikimedia.json), on the development window over the first ten admissible partitions, 501 and 711 examples, none dropped. The adapters' row is their own minus sibling over the same partitions and examples (rank 32). Generated from the artifacts:

Planned before any run: the rules-file comparator

The registered rules-file comparator (registered decisions, "Comparators and audits"), on the retrieval comparator's runner, a rules file in each pool's place. Design of record: docs/superpowers/specs/2026-10-05-rules-file-comparator-design.md. Exploratory, binding nothing.

Planned before any run: the retrieval comparator on the registered runner

The registered comparator (registered decisions, "Comparators and audits"), rebuilt so its contrasts are the adapters' contrasts with a pool where each adapter was. A first draft (#77, 2026-10-02) was amended before any generation, after an independent review found three biases. Its pools were whole halves of unequal size, with a foreign pool twice their size, and closer neighbours in a larger pool pull sibling minus foreign toward zero. It deduplicated each organization once, where the runner deduplicates each half. And it pooled the halves in one bootstrap. None of these survives here.

Power at a test window's realised size implemented

"Fetch horizon" registers that a test window holding fewer changes than its simulation projected is reported beside its verdict with the simulation's power at the realised size. scripts/power_at_size.py computes it. It takes the test report and the simulation the report was read under, and refuses a simulation whose organization, K, projected size, calibration or bounds differ from the report's, a report at or above its projection, and a corpus whose rebuild does not give the simulation's pilot changes and partitions. It reruns the simulation's trials, at its seed, at the number of changes the cell was read on, at the lift that registered each level's bound and at the calibrated run shift. Neither the lift nor the shift depends on the test size, so no calibration or bisection is redone. It records the share of trials supported at each level and its Monte Carlo standard error, sqrt(p(1 - p) / trials), as Morris, White and Crowther (Stat. Med. 2019) ask of simulated performance. Power is at the registered bound, never at an observed effect. Both gates take these artifacts as powers, require one for every organization in below_projection and none for any other, check each Monte Carlo SE against its power and trial count, and report each level's power in the cell's size. partition_sensitivity.py now records its seed, and the pool and partition rebuild both scripts use is one function (pools).

OpenStack and Wikimedia development pilots rerun to name their organization

partition-pilot-openstack.json and partition-pilot-wikimedia.json were rerun by the current partition_pilot.py on the same 22 runs each, in admissible order. A test read sizes K only on a pilot that names its organization (pilot_sizing), and the earlier artifacts did not. The cell, its intervals, its per-run values and the sizing are unchanged, so K stays 24 and 16, and each pilot's per-run spread still matches the calibration its simulation records (same_calibration), as a test read checks. The reruns add the fields the current pilot writes (org, window, p_one_sided, bootstrap_se, meaningful) and record the absent bounds per level.

H1's verdict over the admitted organizations' test reads implemented

h1_test_gate in sphragis/experiment/decomposition.py reads H1 the way replication_gate reads the GitHub family. It takes the organizations from ADMITTED_ORGANIZATIONS, never from its caller, and refuses to run before they are frozen. It also refuses while H2 is confirmatory (Qt and Chromium both admitted), because Holm cannot be read without H2 and H2's test read over partitions is not built. It takes one partition_pilot.py test report and one simulation artifact per admitted organization, exactly those. Each report is checked by require_readable at every registered Holm level and by require_test_read against registered_read, and each bound is read from the organization's own simulation (_registered_bounds, shared with replication_gate). H1 is the intersection-union over the cells at the registered levels through the existing holm_steps, with below_sesoi and the pre-committed reading. With OpenStack and Wikimedia admitted, H1 is read at 95% on two cells.

The 2026-10-04 pass and a sweep of September and October 2026 are now positioned in the papers intake (papers #54), with a watch list of queries per topic for the weekly sweep. Nothing found pre-empts the decomposition (own, sibling and foreign over repeated partitions) or the GitHub replication family. Two RQ2 claims are narrowed (ROADMAP, Plan F). An anonymous ICLR 2027 submission identifies a released LoRA adapter's training document from its weights, so RQ2's claim is source at the organization unit from the transmitted half of a split, not that adapter weights reveal their source in general. A second shows a post-hoc privacy audit can measure the shift between sources instead of leakage; RQ2 therefore attributes between language-matched organizations and reports the same attack between two disjoint draws from one organization beside each result. Owner-approved sharing of agent memory exists in deployment, so the owner-declared boundary is worded as declared per example, by provenance, before training. The Stage 1 report is RQ1 only and states none of these claims. Anonymous submissions are recorded by OpenReview id and cited only once public.

  • in Corpus
  • 2 min read

A test-window read takes its levels, cells and spread target from the registration

What a test-window read still took from the command line (the Holm family's size, the cell count its bound was simulated for, the spread target, and which development pilot calibrated the simulation) now comes from the registration. registered_read(org) in sphragis/experiment/decomposition.py gives a Gerrit organization the Holm levels of design(ADMITTED_ORGANIZATIONS) and its H1 cell count, a GitHub member 97.5% alone and one cell, and both the spread K was sized on (sizing_bound_90, as "Stated power" states the bounds). ADMITTED_ORGANIZATIONS is None until one commit freezes the admitted set before any test window, and a read is refused until its organization is frozen. partition_pilot.py derives these and refuses a flag that differs. The report records the spread targets recomputed from the --sizing pilot (spread_targets in partitions.py, shared with partition_sensitivity.py) and those the simulation was calibrated to, and require_test_read and the gate refuse a pair that differ beyond a relative 1e-9 (the chi-squared bounds pass through libm, whose last bits can differ between the cluster and the reading machine). That ties the two by content without rerunning the simulations. Once ADMITTED_ORGANIZATIONS is frozen, decomposition_gate refuses any other admitted set. require_test_read checks the report's levels, intervals, spread target and cell count against registered_read. replication_gate now takes each member's simulation artifact and reads the bound from it, where it took a bound from its caller and could only check that bound against the one the cell recorded. With OpenStack and Wikimedia admitted and no H2 cell, the family is H1 alone and a test read is at 95%; both levels stay in the simulations, so Qt and Chromium's admission changes nothing that is already computed. Decided here, and written into the Pass rule: the registration described only the two-hypothesis case ("both are read on a two-sided 97.5% interval"), and a review read that as requiring 97.5% even with no H2 hypothesis. Holm with one hypothesis gives it the whole family-wise level, one-sided 0.025, which is what decomposition_gate has computed since it was written (holm_levels(len(confirmatory))); reading H1 at 97.5% then would spend power on a hypothesis that does not exist. Stage 1's analysis section gets the same sentence.