PILOT RERUN: the number to quote¶

Job 143893, same registered model and budget, with every confound the review found removed: deduplicated first (174 of 201 kept), prompt through render_chat for training and both arms, every training example seen exactly twice, LoRA init seeded before attachment. Change-grouped split: 88 changes to 70 train / 18 held out, 147 / 27 examples, 0 of 27 held-out examples repeating a training pair.

Training: 20 of 20 optimiser steps applied, 294 exposures (147 x 2), loss 0.85 to 0.30.

Adapted hits: 11 of 27, spread over 9 of 18 changes. Base hits: 2.

What it says. Exact match moves, and by more than the first run reported, because the first run's held-out set was padded with duplicate misses. Roughly half the exact-match gain is whitespace fidelity: the base model is 0.185 below its own normalized score and the adapter closes that gap entirely. The other half survives normalization, +0.148 with an interval above zero, so the adapter is also getting more edits right, not only formatted. For RQ1 the whitespace half cancels, since both arms are adapters.

Caveats. 27 held-out examples; the interval is wide and the false-positive rate of this bootstrap at 18 changes is about 6% two-sided. One month, one organization, one seed. Built before the acknowledgement filter, so 4 examples carrying only "Done"-type comments are still in it. Still not the RQ1 contrast, which needs the Qt corpus.

Decoding ceiling. Both arms generated at most 96 new tokens. 4 of 172 deduplicated references are longer (99, 114, 143 and 154 tokens), and 1 of the 27 held-out examples was among them, so it was a miss for both arms by construction. It cannot favour either arm and moves both exact-match rates by at most 1/27. The adapted model's longest answer was 54 tokens, so nothing it did produce was cut. MAX_NEW_TOKENS is now 256 for every generator.