Corpus yield, measured on live OpenStack data¶

The first real fetch. Numbers the Stage 1 report's sampling section needs, and which no amount of reading the API docs would have produced.

Correction (same session). The first run counted 4 of 15 comments as "diff errors" and I attributed them to comments on the final patch set. That attribution was never verified: the diagnostic run died on a transient network failure, and a clean re-run of 9 comments found 1 on a final patch set, 8 diffs fetched fine, and zero genuine diff errors. So the final-patch-set condition is real and confirmed, the rate is not established, and some of the original 4 were probably network flakes rather than data. has_successor_revision is correct either way; the number attached to it was not.

The anchoring rule is now a pre-registration decision with a measured cost. The spec says a comment must fall inside a changed hunk between patch set n and n+1. That is the defensible rule, because the edit plausibly addresses the comment, and it keeps 27% of comments. Widening to a 10-line neighbourhood keeps 67%, roughly 2.5x the data, at the cost of a weaker link between the comment and the edit.

Recommendation: keep strict as primary and report the widened number as the sensitivity a reviewer will ask for. Widen only if the pilot power analysis says the strict corpus is too small, so the decision is driven by the minimum detectable effect rather than convenience.

Rough projection at the strict rule: OpenStack's 2,350 merged changes in October 2024 imply on the order of a few hundred examples per month. Verify against a real month before the report quotes anything.