The censoring confound, after two bugs in the inputs¶

scripts/censoring.py, datasets/results/censoring.json. Snapshots select changes by last update; windows assign them by creation. A change created inside a window whose last update falls after the final collected month is absent entirely, and what disappears is the slow reviews. The loss cannot be counted, because a truncated change leaves no record; the lag from creation to last update can be, for changes whose lag was short enough to be seen. That is right truncation, and Lynden-Bell (1971) gives its nonparametric MLE.

The estimator is right. The inputs were wrong twice, and the published figures with them. Review checked the estimator against simulated data from known distributions at n up to 4M, including the case where the largest lag equals the largest horizon, and found a maximum absolute error of 0.0008 scaling as 1/sqrt(N): Monte Carlo noise, no bias. Then it found both things feeding it were broken.

• A Gerrit Change-Id names a family, not a change. It is shared across cherry-picks and relation chains, so restricting the sample by change_id admitted every sibling of an example-bearing change, including siblings carrying no example that settle faster. Qt's sample was inflated 69%, OpenStack's 18%. The key is now (project, change_id, created).
• The last collected month was a shared constant. Qt's examples stop at 2025-09 because the ban blocked 2025-10, so every Qt horizon was one month too long, every risk set held a cohort-month that could not have produced an observation, and Qt's loss came out too small. It is now derived per organization from the months actually built.

Qt's dev figure is not all censoring: its window is 2025-09 and 2025-10, and 2025-10 was never collected, so 70.4% is one absent month plus 41% of the other month's cohort. That is the honest accounting for the corpus as it stands, and better than the earlier number, which silently assumed a month that does not exist.

The decision survives; the numbers that justified it did not. The confound is still the dev window abutting the collection boundary rather than creation assignment, and the test window fetched at in-principle acceptance still carries 0.54 points of differential against the dev window's 31.26, a factor of 58 rather than 26. Creation windows stand, no embargo gap, no last-update assignment. The registered horizon remains: the test window is fetched no earlier than three months after its final month, which holds the differential to 1.16 points.

Three limitations, one of which is a live failure.

• OpenStack's fit is rejected; quasi-independence is not the reason. Superseded by the entry below: the per-cohort rate comparison that first suggested a latency trend is biased by the truncation it is meant to detect, and the conditional Kendall tau rejects quasi-independence for neither organization. Calibrated against its own simulated null, the goodness-of-fit statistic still rejects OpenStack at p = 0.017 and clears Qt at p = 0.580, so OpenStack's fit is off for a reason that remains unidentified. A refit on recent cohorts bounds the effect at about seven points on its dev figure, in the direction that makes the window less censored, and leaves the test window's figure smaller still.
• That check is not fully independent. Where the largest lag equals the largest horizon, the estimator's top step is algebraically the reference cohort's own empirical CDF, so the two must agree at the top regardless. It is a sanity check on the middle of the curve.
• The tail beyond the longest observable lag is unidentified. F(max) = 1 is imposed, so every figure is a lower bound. At an assumed 2% tail the dev figures move to 40.4 and 71.0 and the test window at acceptance to 3.1 and 2.5. The sensitivity used to be computed against a hardcoded ceiling of 1.0 above the support, which put back exactly the tail mass it had removed and understated itself by a quarter to a half; it now carries the fitted CDF's own supremum forward.