Skip to content

A GitHub collection route, and what live pull requests taught it

sphragis/corpus/github.py and github_api.py (spec docs/superpowers/specs/2026-10-01-github-route-design.md): pull requests shaped as Gerrit changes, so build, refine and dedup run on them unchanged and no frozen Gerrit month goes stale; the route's own digest (GITHUB_RULES) stales only GitHub months, failing closed when a GitHub snapshot lacks it. fetch --via github collects a registered organization-month; the sealed test window is refused before any request, as on the other routes.

A live run on eight sampled Apache PRs changed the design twice before any corpus was built:

  • Force pushes. GitHub authors amend and rebase, so the commit a reviewer commented on is often missing from the PR's final commit list: on the first four PRs, 90 of 121 comments had no recoverable commit. Patch sets now come from the PR's timeline (its commits plus every force push's before and after commit). On the eight PRs, 65 of 167 comments are still unplaced, all on one PR whose earlier non-head commits the timeline does not list.
  • Incremental commits. A Gerrit patch set is the author's whole next revision; a GitHub commit is one increment, so diffing the commented commit against the very next one missed fixes made commits later (15 diff_error on the eight PRs). The successor is now the first later commit that changes the commented file, and a file no later commit changes is an empty diff (no_anchored_hunk), as an untouched file is on Gerrit: diff_error went to 0 and examples from 11 to 14.

The sizing's example projections are provisional. They apply the Gerrit thread-to-example conversion (0.217 to 0.282) to GitHub threads; on the eight live PRs, 14 examples came from 167 comments, where the denominator counts every comment (author replies included), not the sizing's reviewer-started threads, so the two rates are not comparable. GitHub's own conversion is read from a built pilot month per organization before any of them is admitted.

Independent review, and the redesign it led to. A reviewer with no part in writing the route found nine problems, two of them in exactly the cases the route exists for: GitHub's compare API diffs from the merge base, so a force-pushed successor's diff started from the base branch, not the commented file; and merges from the base branch passed upstream edits off as the author's response, with no counterpart of the Gerrit routes' rebase guard. Also: timeline order put amended commits backwards, GraphQL rate limits (HTTP 200 with errors) were read as withdrawn PRs, logins survived in code spans and github.com/<login> links, github_api.py was outside the route's digest, splitlines broke lines on form feeds and line separators, and a rename read as an unchanged file. All are fixed and tested, most by one change: each diff is now computed at collection between the two exact file versions with git, carried in the row as NoteDb rows carry theirs, and the build runs offline. The successor is the first commit made after the comment that changes the file; a commented file changed on the base branch between the two commits' fork points drops the comment (upstream_change); patch sets are ordered by commit time and include every commit a comment names. On the same eight PRs: 22 examples (11 before either fix), 0 unplaced comments (90 of the first 121 before), 9 comments dropped for upstream edits that the first design would have built as the author's work.

A second review, of the redesign. A second reviewer found the upstream guard itself wrong in three ways and two earlier fixes partial. After a merge commit every PR commit is reachable from the base branch, so a merge base against its current head is the commit itself and every comment on such a PR read as upstream; the 300-file cap of the compare API (which pages commits, not files) could hide the commented file; and a three-dot check misses a fork point that moved backwards. Also: edits made after the commented commit but before the comment entered the diff; a rebased series shares one committer second; one missing object marked a whole PR withdrawn; and a participant whose login is a word ("fix") had it replaced in prose. Fixed and tested: fork points are now taken against the base branch as it stood before the PR merged (the merge commit's first parent); the guard compares the base branch's own version of the file at the two fork points (no file list, either direction) and drops a comment with a merge commit between it and its successor; a file changed between the commented commit and the comment drops the comment (outdated_view); a same-second series is ordered parent first; a missing object costs only its comments (object_gone) and a persistently failing PR only itself (failed, with its reason); account references are replaced in every structured form (@login, github.com, raw.githubusercontent.com, *.github.io) and bare words are left, as on the Gerrit routes. On the same eight PRs: 18 examples; 17 comments dropped for upstream edits, 6 as outdated views; the four comments the merge-commit fault had dropped on one PR are kept. The 10 drops on one heavily rebased PR were checked: it holds 4 merges from main, and the commented file has three different versions on the base branch across its three fork points.

The first pilot month failed, and what it changed. OpenJDK 2024-11 through fetch --via github stopped after ten minutes: one GraphQL lookup of 50 file versions in openjdk/jdk timed out, and a timeout below the HTTP layer was not retried, so the month's work was lost. Now a timeout or dropped connection is retried with backoff like a server error; a version lookup that times out is halved and retried down to single lookups; a PR whose requests still fail costs only itself. Two costs the failure made visible are also addressed: a PR with no reviewer's inline comment is counted after one request (no_inline_review) rather than walked, since it can yield no example, and each finished PR's scrubbed row is checkpointed, so a month that fails partway resumes.