Skip to content

Apparatus

The GitHub route's digest restored; network-bound digests pinned

# research(2026-10). A comment edit to sphragis/corpus/github.py in #107 (2026-10-08) moved GITHUB_RULES from 56b5374b852d17d7, the digest every GitHub month records, to f89b7378b3675611, so the loader on main refused all of Apache's, LLVM's and .NET's months as "fetched under other GitHub route rules". The collector runs an older checkout and never saw it. github.py is restored to its 6804898 bytes, which brings the digest back; the comment's reading of GitHub's Acceptable Use Policies is the older one, and the 2026-10-08 entry above stays the current word. The loader then reads every Apache, LLVM and .NET month. It is the second time the route's digest moved under collected months (a code change in #101, restored in #102, was the first), so tests/unit/corpus/test_rule_pins.py pins BUILD_RULES and GITHUB_RULES to the digests the collected months record, and FETCH_RULES to the code the git route's first fetch will record; a deliberate rules change updates its pin in the same commit as the rebuild it implies. RULES_VERSION stays unpinned, since a label change is re-refined from disk.

Chromium grants the REST API: its pool becomes H2's foreign source

# research(2026-10). Chromium's infra-dev list answered the narrowed request of 2026-10-15, about 2,000 merged changes with their inline review comments created 2024-11 to 2025-08 from one client at no more than one request a second, on 2026-10-15 (18:28 PDT), from a Chrome Infra address: "There are no concerns with hitting Gerrit's REST API at such a rate." The thread is titled for chromium-review and android-review, and AJ reads the answer as covering both. The request named NoteDb over git; the answer names the REST API, which is the route used here. That scope is a foreign pool, not Chromium as an evaluated organization, whose full corpus stays blocked as on 2026-10-14 (chromium/src's 18,509 merged changes a month against the host's 10,000-result cap).

A REST build can ask only what the host has not answered

A REST build asks the host for each change's comments and, for each anchored comment, a diff, and under Qt's granted interval those requests set how long its build takes. build --fewer-requests asks less: a change whose snapshot records total_comment_count 0 (Gerrit counts every inline comment across a change's patch sets there) is not built and is counted as uncommented_by_count in the month's drops, and each diff is asked once per change however many comments sit on that file and patch set (a failed request is not remembered). The flag is off by default, so every build without it, OpenStack's and Wikimedia's included, is as before; a month built with it says so by carrying the uncommented_by_count key, even at zero.

Planned before any planted quote-style result is read: the exchange rate on a code-level convention

# research(2026-10). δ is carried from exact match to s at the rate of the adapters' gain over the base model on each scale (0.4372 / 0.2384 over 92 own-half development arms, 2026-10-09), and most of that gain is the adapter learning the task's output format, which says little about the rate for small differences between adapters (Stage 1 review panel, item 15). This measures the rate on a convention of the kind the study asks about, fixed here before any run:

Qt grants the REST API in writing; its host joins REST_PERMITTED

Qt's Gerrit administrators answered the 2026-10-14 request by email on 2026-10-15 (QTQAINFRA-8064): requests at a 2-second interval are fine, an identifying User-Agent works for them, and they asked for the address the requests come from, which was sent the same day (the collector's own address; a change is announced before collection resumes). If their automated systems block the client, they unblock it on request. codereview.qt-project.org is now in REST_PERMITTED at those terms: 2 s between requests and one client machine-wide. The 15-minute quiet period after any 5xx is ours, as long as Wikimedia's, since Qt's firewall once outlasted 25 minutes of silence (pacing.Pacer). robots.txt still reads Disallow: /; the written grant is what opens the host.

The matched foreign draw implemented

holdout.matched_foreign_draw builds item 3 of the H2 plan: for one run's sibling draw it takes as many in-language rows from the foreign pool as the sibling holds and fills the rest from the pool's other rows, each stratum shuffled by the run's training seed and the sibling it is matched to (a refinement of item 3's "the run's training seed", so a run's two foreign adapters get independent draws). A pool that repeats an example id is refused, since repeats would count as capacity. A pool that cannot cover a draw is refused, never trained smaller, since that is a source that does not qualify. The partition jobs call it once a source's maintainers grant permission and its pool is built; no foreign draw has been made.

Google's git hosts need their maintainers' written permission, as Qt's REST API does

# research(2026-10). The 2026-10-08 entry admitted android.googlesource.com on its own robots.txt, which leaves git fetching open (RFC 9309 ties a robots.txt to the scheme and authority it is served from, sections 2.3 and 2.3.1.2), and set aside the reading that the review site's Disallow: / expresses the operator's wish about the records themselves. That reading is the stronger one: RFC 9309 says its rules "are not a form of access authorization" (section 1), so compliance with a robots.txt does not settle whether collection is wanted, and Gold and Krinke (EMSE 27, 2022, section 4.2.5) extend a mining study's compliance to a platform's terms and conditions, citing a platform's demand that researchers stop collecting data it had not permitted. The study asked Chromium's infra-dev list on 2026-09-23, mentioning the AOSP data already held, and has had only an automatic reply; AOSP's own maintainers were not asked. So every googlesource git host is refused until its maintainers grant permission in writing: GIT_PERMITTED refuses android, chromium and fuchsia (fuchsia added), and every host's pacing floor now applies whether or not it is permitted.

A test read refuses a run whose outcome-neutral battery did not hold

# research(2026-10). The halt rule ("any outcome-neutral test failing halts the study") was enforced on the unplanted runs only for leakage, which writes halted into a run and partitions.partition_run_windows refuses; a failed positive control, manipulation check or non-degeneracy check was recorded in the run's battery and never read. Registered-report guidance applies a registered exclusion rule by the procedure fixed at Stage 1, never case by case (the TU Delft and JBEF Registered Reports guidelines), so the gate now does it: every run a test read takes must carry a battery that held (neutral.recorded_battery, neutral.battery_holds), the likelihood rescorings through the generation run named in scored_from, and cells.require_test_read refuses a report that does not record one for every run file. A rescoring borrows the battery only of the run it scored (neutral.SCORED_IDENTITY: model, rank, seeds, training size, corpora), and a planted run, check 5's, is refused as one of H1's runs. A development read records each run's battery without refusing. All 46 unplanted development runs resolve a battery that held, so no reading changes.

Wikimedia's K under the coverage rule: 20

# research(2026-10). partition-sensitivity-wikimedia-rp1.0-logprob_per_token-k<K>-null.json (TIGRIS 255027 to 255033, pinned at 642f6d9), null only, 4,000 null trials per K, at the 90% sizing bound; K = 10 is the full simulation above. The rule fixed in the previous entry reads this grid.

Planned before any likelihood result is read: H1 on the reference's log-probability

# research(2026-10). Decided with AJ after the check-5 failure above: H1's primary per-example score becomes the mean log-probability per token of the held-out reference under each arm (likelihood_score.py, logprob_per_token), with greedy exact match kept as a registered secondary. Held-out log-likelihood is the log score, a strictly proper scoring rule (Gneiting and Raftery, JASA 2007), and comparing per-author language models by the perplexity they assign a text is the established attribution method (Huang, Murakami and Grieve, PLOS ONE 2025, authorial language models). It needs no decoder, so it cannot drop a convention learned at a minority rate the way greedy decoding does. Sampling-based exact match was rejected: greedy beats the average sample on most benchmarks and sampling adds variance that can reorder models (Song et al., NAACL 2025), and temperature 0 was best for code refinement in a 2024 replication (arXiv:2412.02789).

The rules files against no file and against each other, at penalty 1.0

# research(2026-10). rules-reading-openstack-rp1.0.json, rules-reading-wikimedia-rp1.0.json. The read contrast, distilled own minus sibling, keeps its reading at penalty 1.0 (the rescoring entry above). The contrasts the 2026-10-07 read reported but did not read move, and three now lie wholly beyond the SESOI. Each is a 95% interval over the first ten admissible partitions: