The open-weight rater is adopted by its rule, and it shows the two model raters agree with each other more than with anyone else¶

label-audit-v2-open-rater.json (labels label-audit-v2-open-rater-labels.json, SPORC job 21795722, six minutes on one A100). Qwen3-14B at pinned weights labelled all 358 items still in the refined corpus, none unparsed (the 25 audited examples a later refinement removed are left out, 13 of them human-checked, so the human pairs read on 139 items). By the rule fixed beforehand it is adopted: its label kappa with the human, 0.254 [0.102, 0.404], reaches the lower end of the locked human~A interval, 0.154. That bar was lenient by design, so the comparators on the same 139 items carry the reading: the human agrees with rater A at 0.289 and with rater B at 0.343, so the open rater sits just below A, its interval overlapping both. Collapsed to valid or not, its kappa with the human is 0.367.

What it shows about the audit. On the same 139 items raters A and B agree with each other at 0.655, about twice what either reaches with the human or with the open rater (0.207 with A and 0.224 with B over all 358). Two raters from one model family agreeing is weaker evidence than it looked: some of that agreement is a shared way of judging, not shared correctness. The audit's conclusion rests on the human's agreement with the raters, which this leaves where it was; what changes is that A~B is no longer read as an independent replication. The audit now has a rater anyone can rerun at fixed weights.