Superseded on 2026-09-23. The share was measured on corpus v1 with Qt's lint bot in the data; the bot audit found nearly half of Qt's organizational effect was the bot. The share on the cleaned corpus is not re-measured in this entry.
Research Computing retired rc-onboard for research jobs on 2026-09-16, and fl-mlm has no
TIGRIS association: in ColdFront both fl-mlm and prdiscourse carry Needs Review, and
the RC documentation grants every project without a pending review a TIGRIS Slurm account.
sacctmgr show assoc lists fl-mlm on sporc only, and sbatch --test-only against the
tigris partition fails with Invalid account or account/partition combination.
Calibrated against its own simulated null, the fit's maximum gap against the longest-horizon cohort rejects OpenStack at p = 0.017 and clears Qt at p = 0.580. The roadmap carried this as an open item with a named suspect. The suspect is innocent, and so are the next two.
sphragis/measure/probe.py, scripts/separability.py, datasets/results/separability.json.
scripts/quasi_independence.py, datasets/results/quasi-independence.json. Lynden-Bell needs
the lag and its truncation limit to be quasi-independent, and the limit here is a
deterministic function of creation month, so the assumption is that review latency is
stationary over the eighteen months the corpus spans. The log recorded that it fails, citing
OpenStack's P(lag<=1) rising from 0.667 in the oldest cohort to 0.905 in a recent one. That
citation was wrong, and the conclusion drawn from it was too strong.
Zhang et al., Test of Time: Rethinking Temporal Signal of Benchmark Contamination (arXiv:2509.00072, ACL 2026), argue that post-cutoff performance decay, the signal the time-partition probe reads, is not dependable evidence of contamination. Their result is that the construction of the items distorts the temporal pattern independently of the source material: model-transformed questions and cloze questions drawn from the very same documents produce markedly different temporal signals.
The contamination battery compares a post-cutoff window against a pre-cutoff control, so the cutoff is load-bearing and had never been sourced. Search returns "June 2024" for Qwen2.5-Coder with some confidence. It does not survive the primary source: in QwenLM/Qwen3 discussion 1093 a user asks whether the cutoff is June 2024, a maintainer pings two team members, neither answers, and a third participant offers June as a personal opinion. No Qwen maintainer states a date. It is a community guess with a citation-shaped shadow, and the battery must not rest on it.
sphragis/experiment/neutral.py, applied locally to rq1-pilot-equalized.json with the
pilot's split recomputed (dedup, change-grouped holdout seed 0, equalized training seed 0).
The recomputed held-out sets are identical to the ones the pilot scored (27 and 133 ids).
Job 143899, scripts/rq1_pilot.py, 16m44s on one GH200. One month per organization
(2024-10), deduplicated, held out by change; one seed; both arms under the chat template.
datasets/results/rq1-pilot.json.
Job 143956, same windows and checkpoints as 143898, scoring each hunk with its three context
lines either side. datasets/results/contamination-openstack-with_context.json.
Job 143898. OpenStack 2024-10 against the 2024-01 control, both deduplicated. Min-K%++ and
Min-K% on the base Qwen/Qwen2.5-Coder-7B; guided completion on the registered Instruct
model. datasets/results/contamination-openstack-after.json.
research(2026-09). The design says the corpus "starts 2024-10-01 against a base model
published 2024-09-17" and calls the time partition the weakest of the three methods
because it rests on a publication date. It rests on less than that.
Job 143893, same registered model and budget, with every confound the review found removed:
deduplicated first (174 of 201 kept), prompt through render_chat for training and both
arms, every training example seen exactly twice, LoRA init seeded before attachment.
Change-grouped split: 88 changes to 70 train / 18 held out, 147 / 27 examples, 0 of 27
held-out examples repeating a training pair.
Seven diagnostic jobs, roughly two minutes each. Worth recording because the first conclusion was wrong and the real one is unintuitive.
warmup_ratioTrainingArguments keeps only warmup_steps. The declared budget keeps the ratio, because
it is scale-invariant and pinning steps would mean a change in corpus size silently changed
the warmup fraction on a pre-registered parameter. training.warmup_steps derives the step
count at construction.