Skip to content

Qt's development pilot on its granted corpus: H1 supported on likelihood, K = 10

# research(2026-10). Qt's 22 development runs, the k-th on the k-th admissible partition at training seed k and N = 1,850, each chained partition_run -> decoder_check -> likelihood_score at the registered decoder (TIGRIS 298699 to 298764, pinned at faa8e8d), read by partition_pilot.py in admissible order (partition-pilot-qt-likelihood.json, partition-pilot-qt-rp1.0.json):

metric runs estimate 97.5% verdict 95% verdict changes (examples) K sized
likelihood, nats per token (registered) 22 +0.0101 [+0.0032, +0.0183] supported [+0.0042, +0.0172] supported 403 (800) 10
exact match (secondary) 22 +0.0028 [-0.0089, +0.0141] inconclusive [-0.0076, +0.0127] inconclusive 403 (800) 13

Every run's outcome-neutral battery was recorded and held, and the aggregation is reproducible. "Supported" is read here without bounds. The K sized is the floor: the formula gives 1.25 runs and runs_needed clamps it to 10. Qt's simulation, which sets its detectable effects and its K under the coverage rule, and its planted check 5 runs are queued on TIGRIS and follow in their own PR. As for the other organizations, the development window does not stand for the test window.

Test window. window-report-qt-v3.json reads the granted corpus; the committed window-report-qt.json (2026-09-23) read the held corpus from before the grant and stays as that record. project-windows-qt-v3.json: 2,951 training changes over 10 months, 0.968 captured, 304.7 a month; the development window predicted at 430 against 409 observed (+5.1%, inside the 15% tolerance, so the check holds); the test window projected at 3,589 changes over 12 months at a 2027-01 fetch.

A continuous simulation reads no placebo. partition_sensitivity.py resamples the pilot's runs on a continuous metric and read the placebo run only to check its decoder. --placebo is now required only for exact match (still checked when given), so Qt needs no placebo run on its granted corpus, and the committed Qt placebo, from the held corpus, is not read.

Qt's generation runs are not committed. Their prediction fields are model text, and a prediction can repeat the example's review comment word for word, which the grant ("no review text is published") and docs/protocol.md exclude. The likelihood rescorings, which carry ids and scores only, are committed and are the runs the pilot reads; the generation runs stay on the cluster and the collector machine, and .gitignore keeps them out of every clone, under a test that no tracked Qt or Chromium result carries a prediction. The likelihood pilot reads their batteries through scored_from, so rerun where they are not held it records each battery as not recorded rather than refusing; the exact-match pilot lists them as its runs and cannot be rerun from the repository at all. Whether the generation runs already committed for OpenStack and Wikimedia keep their predictions is AJ's decision (ROADMAP).