Findings¶

What has been measured, and the script that reproduces each number. The argument these support belongs to the manuscript; this page is the index.

Findings 1 to 3 and 5 were measured with qwen2.5:7b-instruct on an 8 GB RTX 3060 Ti, finding 5 at 600 evaluation tasks. Finding 3b sweeps six models from 3B to 14B, and findings 6 and 10 fine-tune Qwen2.5-3B-Instruct; both need a cluster A100, and finding 10's 600-task evaluation is a 2h39m job on one. Findings 7 and 8 call no model at all: they regenerate the corpus from its seed and review verdicts already committed to results/, so both reproduce exactly and run in CI. Finding 9 is a measurement-design result and retracts an earlier version of itself. Finding 11 calls no model either -- it is a property of the corpus's label structure and of who can read what -- so it also reproduces exactly and runs in CI. Findings 18 to 21 are the same kind: they are properties of an estimator and a protocol over the committed corpus and call nothing. The masked aggregate is bit-identical by construction, being integer; the EM around it is float and runs through libm, so it reproduces to float tolerance rather than bit-for-bit. This project has already retracted an unqualified "bit-identical" twice, and this is the scope that survives.

Every artifact in results/ carries the version, commit, platform, model, and seed behind it, so any number here traces back to the run and the machine that produced it. That matters because the model-dependent numbers are not bit-reproducible across machines, while the gate is.

Status: provisional¶

The testbed is still being built, and these numbers should be read as provisional rather than settled. That is a deliberate position, not modesty, and the history supports it: of the first three findings, two did not reproduce when remeasured at larger n on a corrected corpus, and the third had to be retracted outright after a generator bug turned out to have made two thirds of the positive class unanswerable.

What that means for a reader:

• Mechanisms are the durable part. That leave-one-out under-attributes because corroborated facts have no single load-bearing source, or that eligibility is bimodal on the compartment-shedding ruling, survived remeasurement. The severities attached to them did not.
• Every number carries its own verdict. pharos.validity inspects each measurement for the conditions that make a score misleading -- small n, a class floor the score does not clear, degenerate predictions, unparsed answers -- and marks the artifact quotable: false when it trips one. Most of the sweep results are currently not quotable, and say so.
• Numbers below are traceable, not authoritative. Each artifact in results/ records the version, commit, platform, model, and seed behind it, so a figure can be checked rather than trusted.

The gate's calibration finding is the one result here with support from outside this generator, having been reproduced on three public corpora. Treat the rest as the current state of an instrument still under construction.

The findings, by what they are about¶

Grouped for navigation only. The numbering is chronological, so a group's members are scattered through it, and nothing in the grouping is a claim about how they relate.

What these sizes can resolve¶

scripts/measure_power.py, results/power.json

Model-dependent findings here run anywhere from 24 to 600 tasks, and that spread is the point rather than an inconsistency: the effects claimed differ by an order of magnitude, so the size each one needs does too. This prices every claim instead of arguing about a single convention, and the two claims it flagged as underpowered have since been rerun at the size it named. The interval machinery is the same one the measurements use, and the class balance is read from the corpus rather than assumed.

The second column is the number that matters: two conditions separate only when the gap between them exceeds roughly one half-width on each side, which is exactly how uncertainty.resolves decides.

Every headline claim against the size it was actually run at:

A claim measured against a fixed reference is cheaper to resolve than one comparing two measured conditions, and the table prices them differently. The majority floor, the stated-rule ceiling, and an adversary's guessing prior are all computed exactly from a generated corpus, so they carry no sampling noise of their own and the gap has to clear one half-width rather than two. Charging them double was an error in an earlier version of this section, and it understated two claims.

8 of 11 resolve at the size they were run.

The pattern is the useful part. The claims this project makes strongly rest on large gaps and are comfortably resolved; the ones that are not are mostly ones it already declines to make. Two deserve naming:

• "8 shots beats 0 shots" was bought, and the claim died. This table used to say it needed n≈600, an order of magnitude more than the 30 it was measured at. We ran it. The lift is not there: finding 5 reports 8 shots at 0.485 against a zero-shot 0.481, a gap of 0.004 that would need more than 2000 tasks to separate, and the row above records the refutation rather than deleting it. The companion claim, that 2 shots is worse than 0, is the closest thing to a resolved difference here and still is not one.

That bullet is why this section is not a limitation. A static benchmark with 30 items is stuck with 30; a generator is limited only by compute, and 600 tasks is roughly two hours of local inference. The table is therefore a purchase order rather than a disclaimer, and the two purchases it has prompted so far both ended in a retraction. That is the instrument working: an unresolved claim is not a quiet claim, it is a claim with a price tag on it.

Measurement health¶

pharos.validity inspects every measurement for the conditions that make a score misleading and marks the artifact unquotable when it trips one. That flag used to be computed, warned about on the console, and then read by nothing, which made it a console message rather than a property of the record.

It is published here instead of policed. A guard forbidding prose from quoting a flagged number would be wrong: quoting one as evidence that something failed is exactly what the flag asserts, and is what finding 3b correctly does with six models that do not clear their own majority floor. What the flag forbids is quoting such a number as evidence of capability.

This table is generated from results/ and CI fails when it drifts from them.

27 of 61 assessed artifacts are flagged. A flagged number may still be quoted as evidence that something failed, which is what the flag asserts; it may not be quoted as evidence of capability.

Carrying no validity assessment, which is a gap rather than a pass: corpus_sensitivity, estimator_initialization, governance_sensitivity.

Exempt, because there is no sampling question to answer:

• adapter_replication -- compares assessed adapter artifacts against their own replicates; the question is whether two runs agree, which no sampling flag answers
• external_gate_validation -- carries its own permutation-null statistics per corpus
• federation_eligibility -- deterministic over the label lattice; nothing is sampled
• fl_benchmarks -- sizes the problem rather than settling it, is quoted nowhere in the manuscript, and reports a bootstrap interval per condition instead of a flag
• fleet_sensitivity -- a sweep over a nuisance parameter; reports invariants, samples nothing
• gate_determinism -- reports the gate's surface baseline at full precision on one machine; the result is the comparison against another machine, not the number
• gate_determinism-cluster -- the second machine of that comparison
• guard_mutations -- four deterministic edits, each either noticed by the suite or not; there is no population the four are drawn from and no n a flag could be computed against
• power -- prices hypothetical evaluation sizes; simulates outcomes rather than measuring any
• teacher_fleet -- aggregates assessed adapter artifacts; adds no measurement of its own
• triage_lift -- superseded by the per-model triage_lift-* artifacts, which are assessed

1. Leave-one-out attribution cannot produce a correct governed label¶

scripts/measure_label_fidelity.py

Eight-source summarization turns under exact leave-one-out, 360 model calls. A wrong governed label on 7 of 40 turns (18%): 4 leak and 3 creep. Recall is what the ablation misses; precision is what it wrongly blames.

The cause is corroboration. Leave-one-out asks which single source is load-bearing, and a fact reported through several channels has none: drop any one copy and the fact survives in the others, so no source is blamed and none of their labels enters the join. Corroboration is not an edge case in this domain, it is what channels are for.

Read the precision alongside the recall. At 0.942 the ablation rarely blames a source that did not contribute, it simply misses many of the ones that did. That asymmetry is why the failure is mostly leak, and why a tenth of turns (4 of 40) receive a label that under-protects their sources.

Leave-one-out is also the ceiling that cheaper estimators approximate, so nothing faster repairs it. At 67% exactly correct it is not a usable labelling mechanism, though the original "rules out the whole family" was stated more strongly than 24 turns can support.

The replacement costs nothing: given what the output asserts, join the labels of every source that could have asserted it. One detection pass, no ablation sweep, and conservative by construction, so the error direction is creep rather than leak.

2. The design is bimodal on one policy ruling¶

scripts/measure_federation_eligibility.py

Three aggregator ceilings, four capacities, 40 turns. Turns average just under two compartments of 4, and most already sit high on the level ladder, because a summary over eight sources joins nearly everything.

So "may a low-capacity verdict shed the compartments of its sources?" is not a detail. Answer no and the fleet is a set of unconnected local learners. Answer yes and verdict-shaped outputs federate completely while prose never does, which reproduces the design's split table from measurement rather than assertion.

See the label lattice.

3. A corpus bug, a retracted finding, and a real benchmark target¶

scripts/measure_triage_lift.py

This one went wrong before it went right, and the sequence is the useful part.

The first measurement said the specialist could not do its own task: triage accuracy 0.35 against a majority floor of 0.725. The conclusion drawn was that a 7B model cannot evaluate a three-way conjunction over facts split across reports.

That conclusion was wrong, and the cause was a corpus bug. Only 34% of significant events actually rendered all three of their defining facts, because channels were chosen before coverage was checked and any shortfall was padded from the whole vocabulary. Two thirds of the positive class was unanswerable from its own prompt. The model was being scored on evidence it was never shown, and its escalate-on-anything behaviour was a reasonable response to partial evidence.

The shortcut gate could never have caught this: a surface probe tests whether shape predicts the label, not whether the label is derivable from the content. Semantic integrity needs its own checks, and they are now in the test suite.

On the corrected corpus:

The model does the conjunction perfectly when the rule is given and the prompt structures the check. Which exposed a design error: stating the rule leaves nothing for the fleet to learn, and learning the analyst's rule from accept/revise/reject is the entire premise of personalization.

Withholding the rule gives the benchmark its proper shape. A ceiling of F1 1.000, known reachable, against a base of 0.537 to 0.688 that over-escalates at recall 1.000 and precision 0.37 to 0.52. That gap is the target.

3b. Over-escalation is universal, and scale does not fix it¶

scripts/sweep_models.sh, scripts/compare_models.py

Finding 3 was measured on one model, so its central observation could have been a fact about qwen2.5:7b-instruct rather than about the task. Six models, three families, 3B to 14B, 40 rule-withheld tasks each:

Recall is 1.000 everywhere. Not approximately, exactly, including at 14B. Every model escalates every significant event and also escalates most routine ones, at precision between 0.389 and 0.609. The over-escalation reported in finding 3 is a property of the task under a withheld rule, not an idiosyncrasy of one model. This is the one claim that has now held across two different corpora.

Two of six clear the majority-class floor, and three are unresolved. qwen2.5-3b at 0.775 and llama3.2-3b at 0.675 sit above their floor of 0.650, and qwen2.5-3b's interval clears it outright. For qwen2.5-14b, llama3.1-8b and mistral-7b the point estimate is below the floor and the interval reaches it, so the supportable statement is this sample does not show them clearing it rather than that they do not. Only qwen2.5-7b is below it with room to spare. A reader comparing any F1 here to 0.5 would still badly overstate what is happening, which is why both floors are reported.

Scale does not help, and this controls for family. Within Qwen alone, 3B scores 0.775 and 14B scores 0.625: a 4.7x parameter increase that makes things worse, and by more than it did on the previous corpus. The 14B run needs more than 8 GB of VRAM, which is the whole reason it is in the sweep.

That is still consistent with finding 5 and still strengthens it. A model that reaches F1 1.000 when handed the rule is short of neither capability nor parameters. It is short of the rule, so rule acquisition is the whole question.

These numbers are not bit-reproducible, and the gate is¶

Worth stating precisely, because the two halves of this repository behave differently and the difference is by design.

Re-running the five smaller models on cluster hardware changed 2 of 200 judgements: qwen2.5-3b and llama3.1-8b each flipped exactly one task of forty; the other three were identical. Temperature is 0 and the seed is fixed, so this is runtime and quantization numerics moving a borderline verdict, not sampling.

The corpus reproduces bit-identically across the same two machines -- the SHA-256 of the serialized corpus agrees at every seed. The gate's score does not, which we report because scripts/measure_gate_determinism.py measured it rather than assuming it: two of seven seeds disagree, by 1.2e-05 and 6.5e-06, with numpy, scikit-learn, scipy and the BLAS all at identical versions. The BLAS selects kernels by processor -- AVX-512 on the cluster's Xeon, AVX2 on the laptop's Ryzen -- which changes the reduction order inside the probe fit.

What matters is that no verdict moves: the largest disagreement is about four orders of magnitude below the gap between any measured baseline (0.638--0.660) and the 0.720 acceptance ceiling. That asymmetry is still the reason model calls are confined to pharos.attribute, but the accurate form of it is that the acceptance decision does not drift with the platform, not that the number is identical. The scores measured on the corpus carry roughly a one-percent floor on cross-platform agreement. Quote model-dependent numbers with the platform named.

4. Answerability and surface non-leakage pull against each other¶

Fixing coverage raised the surface baseline from about 0.55 to 0.63-0.67, and widening channel sets did not bring it back down. Once every fact of an event must appear across a fixed number of reports, report composition is tightly determined by the event's fact set.

A real tension rather than a bug. It argues for treating the surface baseline as a published property of a correct corpus rather than a defect to drive to chance.

5. In-context learning does not close the gap¶

scripts/measure_rule_learnability.py, results/learnability.json

The design's premise is that a fleet learns analytic craft from an analyst's accept, revise, and reject decisions. Withholding the rule and supplying labelled examples instead is the cheap form of that question.

600 evaluation tasks per condition, one pass each, qwen2.5:7b-instruct.

Examples close none of the gap to the ceiling, and no condition comes near its own majority-class accuracy, which sits at 0.685. The floor is not close: the best condition is 0.20 below it, and on the corrected corpus every condition now sits below the majority floor rather than merely short of the ceiling. Every row is marked unquotable by pharos.validity for exactly that reason, which is the correct reading: none of these numbers is evidence of capability.

Eight shots against zero is +0.003, which is the finding. Their intervals overlap almost completely, [0.444, 0.527] against [0.441, 0.523]. This is the third independent measurement of that comparison and the second on which the lift is absent; the one run that showed a lift was measured on the pre-fix corpus and is discussed in 5b.

Two shots looks worse than none, and we decline the comparison anyway. The gap runs the opposite way to what a reader expects, 0.433 against 0.482, and it is the closest any pair here comes to separating, at 0.049 against overlapping intervals.

That comparison has now returned three different verdicts across three runs, and the reason to decline it is measured rather than argued. One run separated it by a margin of 0.0013 -- the pair cleared its own criterion by roughly a thousandth. Another put the gap at 0.0551 against a half-width of 0.0576 and the criterion returned false. This one does not separate at all. A test whose verdict flips on a quantity far smaller than the measurement's own run-to-run variation is not testing anything, and "do two examples hurt" has changed sign twice on data differing only by which afternoon it was collected and which corpus generator produced it.

We therefore report the direction and decline the difference, which is what the earlier run's arithmetic said and what this run's replication says for a better reason.

That distinction is worth spelling out because two criteria disagree on this pair and only one of them is right. Neither interval covers the other's point, which is what uncertainty.resolves tests, and on that basis an earlier draft of this section claimed the comparison as resolved. But "do two conditions differ" is a question about an interval on their difference, and that interval is wider than either input. uncertainty.resolves_difference applies the correct rule and returns false here. The weaker test was mislabelled "conservative" in its own docstring; it is the permissive one, and it has been corrected.

What the other columns show is a coherent mechanism for the direction, whether or not the size is resolved. Recall climbs from 0.899 at zero shots to exactly 1.000 at eight, while precision barely moves --- 0.368 to 0.380 --- so the examples are not teaching the rule. They are teaching the model to escalate more: false positives rise from 290 to 307 while false negatives fall from 19 to 0. That is the same over-escalation finding 3b found in every model tested, and supplying examples does not repair it.

No other pair separates under the difference test. Four, eight, and zero shots are indistinguishable from each other at this size.

5b. The intervals above understate the uncertainty, measured by replication¶

This measurement has now been run twice, end to end, and the two runs disagree by more than the intervals allow. That is a fact about every model-backed number in this document, not just this one, so it is recorded here where the evidence is.

The two runs used the same 600 evaluation tasks, verified identical by task id and ground truth, the same model, and temperature 0.0 with seed 7 on both. They may not have used the same prompts, for the reason given above.

Two things were observed. Neither is attributed, for the reason above.

A 95% interval did not cover the other run's point estimate. At four shots, Run A reported [0.475, 0.557] and Run B's point is 0.4523, below the bottom of it. Read as decode variation this would say the interval understates uncertainty; read as a corpus difference it says only that two different renderings score differently, which is unremarkable. The runs cannot distinguish these.

The ordering of the conditions differs. Run A puts eight shots at 0.5142, below its zero-shot 0.5234; Run B puts eight shots at 0.5492, above its zero-shot 0.5126. Same ambiguity: a genuine reversal between identical runs would be the stronger form of the power argument, and a reversal between differently-worded corpora would be a fact about two corpora.

5c. The generator defect behind 5b, and the fix¶

src/pharos/generate.py, tests/test_generate.py

The confound named in 5b was a defect in the generator, not a fact about decoding. It is recorded here because the manuscript's self-audit section cites this page as the long-form record of it.

generate() drew every event before rendering any of them, from one random.Random(seed). Event drawing consumes a variable number of values, so rendering began at a stream position set by n_events. Measured against the pre-fix generator at seed 7:

No report was worded identically across two corpus sizes, at any pair. The checks a reader would think to run -- do the task ids line up, does the ground truth agree -- pass at 100%, which is why this survived as long as it did. Any two model-backed measurements taken at different --events were comparing different corpora.

The fix derives a stream per event from sha256(f"{seed}:{index}:{purpose}"), so a smaller corpus is an exact prefix of a larger one, field for field including text. tests/test_generate.py::test_a_smaller_corpus_is_an_exact_prefix_of_a_larger_one asserts every field; it fails on the first report against the pre-fix generator.

5d. The controlled replication, and what it settles¶

results/learnability.json, results/learnability_replication.json

Runs A and B could not separate decode variation from a corpus difference. Runs C and D can, because the generator defect that confounded A and B is fixed and both were run at --events 800, seed 7, 600 evaluation tasks, temperature 0, on commits whose corpus-defining code and prompt path are byte-identical (git diff over generate.py, labels.py, world.py, tasks.py, scenarios/ and measure_rule_learnability.py is empty between them). The only difference between C and D is when they ran.

Reproduction is per-condition, not global. Four and eight shots return bit-identical confusion matrices -- 183/330/74/3 and 188/307/101/0, every cell -- while zero and two shots move by more than two points. Finding 9's claim that a single-pass measurement reproduces exactly is therefore true of half these conditions and false of the other half, in one run of one script.

The two that moved are exactly the two whose unparsed count changed, and the verdicts moved with them: at zero shots, 14 items crossed from false positive to true negative on top of the 4 that became parseable. Four conditions is not enough to call that a mechanism, and it is enough to say the variation is not uniform decode noise -- it is concentrated in the conditions where the model's output format was itself unstable.

What this does not say. Two runs bound nothing tightly, and this is two runs. It establishes that variation at this magnitude occurs, not its distribution. The six-point precaution stated below was set before this measurement and survives it comfortably; the useful sharpening is that the variation is condition-dependent, so a per-condition interval computed within a single run does not capture it.

What survives untouched. The finding is that in-context learning does not close the gap, and every value in both runs sits between 0.45 and 0.55 against a stated-rule ceiling of 1.000 and a majority floor of 0.693. A ten-point band is irrelevant to a claim about a fifty-point shortfall, whichever source it comes from. This section was never load-bearing for the finding; it was an attempt to strengthen a caveat, and it overreached.

The precaution stands even though the evidence for it does not. Whichever source produced these differences, both are live in any model-backed comparison here, so the rule is unchanged: a gap under roughly six points between two model-backed conditions is unsupported without replication at a recorded corpus size. Two claims sit in that range and both were already declined on other grounds: finding 5's shot-count ordering, and the six-model ordering in finding 3b, which the text refuses to present as a ranking. The claims this project does make -- recall exactly 1.000 in every model, a fleet inheriting its teacher's error rate to within 0.002, a consensus cliff from 1.000 to 0.660 -- are all far outside that band.

Three caveats, because this result is still easy to over-read:

• Unparsed answers are not evenly distributed. 10 of 600 at four shots against 4 at eight, and an unparsed answer is carried separately rather than scored as wrong. The variation is unexplained and is a reason to treat the 4-versus-8 ordering as noise even beyond what the intervals say.
• Eight shots is roughly 1,300 words of examples before the target case, so long-context dilution is an unseparated confound.
• In-context learning is not gradient learning. Eight examples in a prompt and a LoRA trained on 1,140 tuples are different mechanisms, and the first failing does not establish that the second will -- finding 6 shows it does not.

The useful conclusion is about sequencing, and it is now stronger than when the lift looked real. The cheap proxy for the design's central premise came back negative, not merely inconclusive, so the premise cannot be validated cheaply and the adapter experiment is the thing that decides. What the ceiling establishes is where the bottleneck sits: a model that reaches F1 1.000 when told the rule is not short of capability, it is short of the rule, so rule acquisition is the whole question.

6. Gradient learning does close the gap, on clean labels¶

scripts/train_adapter.py, results/adapter_learnability.json

Finding 5 ended by saying the adapter experiment was the thing that decides. It has now run: a LoRA fine-tune of Qwen2.5-3B-Instruct on an A100-40GB, rank 16, three epochs over 1,140 training tuples, evaluated on 60 held-out tasks.

The rule is gradient-learnable where in-context examples reached none of it. That is the substantive result, and it is the cleanest contrast in this document: same withheld rule, same task, same ceiling, and the mechanism is the only thing that changed.

Four things that keep this honest:

• The split is clean, and this was verified rather than assumed. One task per event, so an index split is an event split: zero event overlap, zero exact-text overlap between evaluation and training, and zero duplicate texts corpus-wide. A perfect score is the shape of a leak, so the leak was the first thing checked.
• The score is saturated, and saturation is not the same as certainty. Zero errors in 60 bounds the true error rate only at 5% by the rule of three, and a ceiling cannot rank anything above it. pharos.validity now flags this automatically; it did not before, and the gap was found by this run passing every existing check while plainly deserving a caveat.
• The base row is worse than it looks and should not be quoted alone. It sits below its own majority floor and left 8 of 60 answers unparsable, both of which the validity checks flagged at runtime. It is a fair starting point for a delta, not a measurement of the model.
• Clean labels are not analyst decisions. The training signal here is the generator's ground truth. The design's actual premise is learning from accept, revise, and reject -- indirect, noisier, and far less abundant. This result removes the capability objection and leaves the supervision question untouched, which is the next experiment rather than a conclusion of this one.

7. Review is abundant; what it costs is correctness¶

scripts/measure_analyst_review.py, results/analyst_review.json

Finding 6 removed the capability objection and left the supervision question untouched: the rule is learnable from clean labels, and the design's premise is learning from an analyst's accept, revise, and reject decisions instead. This measures what that substitution costs, at the boundary rather than only on the verdict.

Eight reviewers, each a parameter grid entry in pharos.analyst differing from the by-the-book reviewer on exactly one axis, review the 40 triage proposals each of the six sweep models actually produced -- 1,920 decisions in all. The proposals are real; the reviewers are not, and the caveat at the bottom of this section is the load-bearing one.

Targets: share of decisions handing the learner a verdict to train on. Correct: share of those targets matching the world. Escalated: share of decisions passed to an authority rather than settled. Recovered: share of the 21 unreleasable proposals whose correction clears the ceiling unaided. Addressed: share either recovered or escalated. The by-the-book row is a control, not a result: its threshold is the world's own rule and it never slips, so 1.000 is what it is defined to produce.

"Far less abundant" was wrong, and this project wrote it. The README and finding 6 both described analyst decisions as scarcer than labels. For a binary verdict they are not: an acceptance is a target, a revision is a target, an escalation carries one too, and only a bare rejection costs anything. A reviewer who revises just a quarter of the time still supplies a usable target on 0.83 to 0.88 of decisions. Scarcity is the wrong worry.

The targets are the reviewer's opinion, not the model's. Target accuracy is identical across all six models -- not close, identical, because a corrected verdict is the reviewer's own call and an accepted one is a call the reviewer agreed with. Review does not import what the model knew; it exports the reviewer's standard. That is the mechanism behind the next row, and it is why the number is a property of the reviewer alone.

A reviewer with the wrong threshold teaches a degraded rule, and the worst of them teaches one that loses to guessing. At threshold 2 the target stream matches the world on 0.750, at threshold 1 on 0.475. The majority floor on this evaluation set is 0.650. Threshold 1 is below it, so a learner trained on those corrections would do worse than always answering ROUTINE -- and it would do so while every acceptance and revision looked like ordinary supervision. Threshold 2 sits above the floor and still costs a quarter of its targets, which is supervision that is worth having and quietly wrong. This is not a hypothetical reviewer. Threshold 1 is over-escalation, and finding 3b measured over-escalation in every model tested, recall 1.000 at precision 0.389 to 0.609. A fleet learning from reviewers who fail on the axis its models already fail on does not get corrected; it gets confirmed.

Unnamed grounds cost nothing in volume and everything in attribution. The unexplained reviewer accepts and revises exactly as often as by-the-book, and supplies exactly as many correct targets, so no yield statistic distinguishes them. What it withholds is which of the two assertions was wrong. A proposal is a verdict and the label it would be released under, so an objection that does not say which one failed asks the learner to fix an error it cannot locate. That axis exists because the output is governed; a preference over ungoverned text has nothing corresponding to it.

The disclosure boundary moves, but only by an authority, and the toll is half the stream. This paragraph replaces one that was wrong, and the correction is the most useful thing in this finding.

Of the 40 proposals, 21 carry a compartment the aggregator is not cleared for. No keep-compartments reviewer's own correction clears the ceiling -- recovery is 0.00, exactly as first measured, and that part reproduces. The first version concluded from it that review cannot move the disclosure boundary. That conclusion was an artifact of the instrument: shared_eligible returned a boolean, so a reviewer had two doors, and the option a real analyst reaches for first was not representable.

Given a third disposition, all 21 are authorizable: the block is a compartment shortfall, which finding 2 had already identified as a policy act rather than an engineering problem, and a policy act is what an authority exists to perform. Every escalating reviewer addresses 1.00 of the blocked set. The no-escalation row differs from by-the-book on that one parameter and addresses 0.00, so the whole of the original claim sits in that parameter.

What replaces it is a cost rather than an impossibility. 21 of 40 decisions become escalations -- 52.5% of the review stream lands on whoever rules on compartments, for every reviewer applying the fail-closed default. The boundary is movable; what a fleet has to budget for is the authority's queue. The reviewer who sheds compartments escalates nothing and recovers everything, which is the same bimodality finding 2 reported, now with its price visible on both sides.

Ensemble agreement over the four actions is Fleiss' kappa 0.435 to 0.450 across the six models. That number is a property of the grid this project chose and should not be read as an estimate of anything.

8. Being right and sloppy beats being wrong and careful¶

scripts/measure_review_sweep.py, results/review_sweep.json

Finding 7 reported target accuracy for eight named reviewers. Each was one point in a space, and a point cannot say where the edge is. This sweeps the two axes that move it, over 40 tasks and five review seeds per cell.

The two axes differ in kind, and separating them is the whole point. A reviewer's standard is how many of the three defining facts they require; getting it wrong is a systematic error, reproducible and patterned. Their carefulness is how often they fail to apply their own standard; getting that wrong is random error.

Target accuracy against the world, * marking cells that clear the 0.650 majority floor:

The most wrong standard never clears the floor. Every cell in the needs-1-of-3 row sits below 0.650, including where the reviewer never slips at all.

A correct standard tolerates a great deal of carelessness. It clears the floor through a 30% slip rate (0.685) and only fails at 40%. The exchange rate is still lopsided, and slightly less so than previously measured: one step of standard error costs what a 30% slip rate costs (a two-of-three reviewer's 0.750 is what the correct standard scores at 30% slip), and two steps cost more than any slip rate measured up to 50%, since the needs-1-of-3 row never reaches even the correct standard's worst cell.

That is the quantitative form of a qualitative claim in the learning-from-noisy- labels literature -- that systematic annotator bias damages a model more than random noise, because the errors are class-dependent and get learned rather than averaged away. The direction reproduces on both corpora. The magnitude is what the exchange rate above states and nothing more.

What this means for a fleet. Recruiting effort should go to establishing that reviewers hold the right standard, not to keeping them attentive. The usual annotation-quality apparatus -- attention checks, redundancy, inter-rater agreement -- measures carefulness, and every reviewer in the top two rows would pass all of it while agreeing perfectly with themselves. Agreement is not correctness, and here it is not even correlated with it.

9. A measurement that repeats one prompt measures the wrong thing¶

scripts/measure_decode_stability.py, results/decode_stability.json

The original probe called one prompt several times in a row and counted disagreements. A real measurement calls each task once, in sequence. Those are different experiments, and only the second one is the reproducibility question.

Single-pass measurements reproduce exactly. Two complete sweeps, each task called once in the same order, agree on every task in both decode regimes, and produce identical accuracy.

The 13.3% is a warm-up transition, not noise. Following it produced the mechanism: in every varied case the first call against a prompt differs and every call after it is byte-identical. Five repeats of a summarization prompt gave [A, B, B, B, B] on 3 of 3 varied prompts. So repeating one prompt measures the cold-to-warm transition of a prefix cache and reports it as irreproducibility. A real measurement never repeats a prompt, so every call is cold, and cold calls agree with each other.

What this cost, and what survives. The retracted version was cited in a second place before it was checked: finding 5's remeasurement warning attributed its caution to a 10% instability that does not exist. What survives the correction is worth keeping:

• The between-task interval is real and was missing. Thirty tasks is a small sample whatever the decode does, and finding 5's conditions genuinely do not separate (see its table). That conclusion rests on sampling uncertainty across tasks, which the cluster bootstrap measures correctly and which no amount of reproducibility removes.
• Finding 3b's correction stands entirely. It was computed from single-pass artifacts by bootstrapping over tasks, and single-pass artifacts are now known to be exactly reproducible.
• The cross-platform figure in the manuscript stands, and its original reading was right. Two of 200 judgements changing across machines is a platform effect, measured under a decode that reproduces perfectly on one machine.

What generalises. Not a claim about temperature or decode length, but about measurement design: a probe that repeats one input is not measuring what a sweep over many inputs will do, and if the backend caches anything, the two differ systematically rather than randomly. That is worth checking before reporting either.

10. A fleet learns its analyst's standard, not the world's¶

scripts/train_adapter.py --reviewer, cluster/review-adapter.sbatch, results/review_adapter-*.json

Finding 6 trained an adapter on the generator's ground truth and reached F1 1.000, which settled capability. Finding 8 then showed a reviewer's target stream can be systematically wrong and that no amount of carefulness repairs it. Neither said what a model does with such targets. This does.

Four LoRA adapters on Qwen2.5-3B-Instruct, A100-40GB, 1,140 training tuples each, identical except for who labelled them. Every adapter is scored twice on the same 60 held-out tasks and the same decode: once against the world, once against its own teacher's answers.

Accuracy on 600 held-out tasks, 1140 training tuples per adapter. The last column is the adapter's agreement with the world minus its teacher's: near zero means the adapter reproduced its teacher's error rate rather than diluting it.

Accuracy on 600 held-out tasks whose events are disjoint from training. The base model scores accuracy 0.320 with 82 of 600 answers unparsable, so every row is a large gain over not training at all. The last column is the adapter's agreement with the world minus its teacher's, and it is the cleanest statement of the finding.

A wrong standard is inherited almost exactly. The two systematically-wrong teachers hand over their error rate to within 0.008 and 0.020: a teacher whose targets agree with the world on 0.701 produces an adapter agreeing on 0.693, and one at 0.447 produces 0.467. The adapter did not partially absorb the reviewer's rule or average it against its prior; it learned "two of three" and "one of three" and now agrees with the world exactly as much as that rule does. This is the concrete form of what the noisy-annotation literature calls absorbing an unreliable annotator's errors into the parameters, and at 600 tasks the correspondence is close enough to read as an identity rather than a tendency.

Random error is filtered; systematic error is not. The contrast is the finding, and the careless teacher is the only row that breaks the inheritance pattern. It slips 15% of the time, its targets match the world on 0.855, and its adapter matches the world on 0.902 while matching the teacher on only 0.790. It tracks the underlying rule more closely than the teacher that taught it. Given a consistent wrong rule instead, the adapter simply became that rule.

So the two error types finding 8 separated at the level of the target stream stay separated after training. Carelessness is partly repaired by learning, by nearly four points. Wrongness is preserved by it to within two points, which is the more unsettling half of the result: there is no dilution, no regression toward the model's prior, and nothing in the training signal that notices the rule is wrong.

What this means for the design. The premise is that a fleet learns analytic tradecraft from analyst decisions. It does -- exactly, faithfully, and without any check on whether the tradecraft is correct. Personalization works as specified, and that is the risk: an over-escalating analyst produces an over-escalating agent, and over-escalation is what every model in finding 3b already does. The mitigation is not more data. It is annotator-reliability weighting, which needs annotator identity retained through training -- something Pharos has by construction and most annotation pipelines discard.

It survives a corpus the adapter has never seen any part of. The result above evaluates on held-out events from the same corpus. A stronger test evaluates on a different corpus instantiation entirely, sharing no events, no vessels and no renderings, which is what contamination-resistance guidance asks of a benchmark. Repeating the sweep with the evaluation drawn from seed 101 (2h54m, same A100):

Accuracy on 600 held-out tasks, 1740 training tuples per adapter. The last column is the adapter's agreement with the world minus its teacher's: near zero means the adapter reproduced its teacher's error rate rather than diluting it.

Both conclusions hold on a corpus with zero overlap. The systematically wrong teachers are still tracked closely, now within about two points rather than half a point, and the careless teacher is still the only one improved upon: 0.943 against its teacher's 0.823, a gap of 0.125 against a difference half-width of 0.038, resolved on its own terms.

At fleet scale: 24 teachers, both axes crossed¶

scripts/measure_teacher_fleet.py, results/teacher_fleet.json

Four teachers is four case studies, and two of the three claims above are ones four points cannot separate: the named teachers vary standard and carefulness together, so "systematic" and "random" are never independently manipulated. The grid crosses three escalation thresholds with eight slip rates and trains all 24, one A100 job per point. It also runs on the corrected generator, so unlike the tables above it is directly comparable with every other finding on this page.

† marks the 12 adapters the validity check refuses, for accuracy beneath the majority floor or for recall bought with more false positives than true ones.

The two conclusions survive, and one number does not. Teachers who never slip hand their disagreement over at a median of +0.0008 -- the inheritance claim, now on three independent points rather than two. Teachers who do slip are improved upon at a median of +0.034, with 12 of 21 adapters ending closer to the world than the teacher that taught them. What does not survive is "close enough to read as an identity": across 24 teachers, adapter-teacher agreement runs from 1.000 down to 0.473 with a median of 0.781. Four teachers could not have distinguished a tendency from an identity, and the grid does.

The apparent decay is a ceiling, not a failure to inherit. A teacher slipping at rate s disagrees with its own rule that often, so an adapter that had learned the rule perfectly would still match those labels only 1-s of the time. Measured against that bound the residual has median -0.030 and never exceeds 0.078 in magnitude. Equivalently, in 20 of 24 cases the adapter agrees with its teacher more closely than the teacher agrees with itself on a second pass, by a median of 0.041. The adapter learns the teacher's rule with the teacher's noise averaged out -- what the weak-to-strong literature calls convergence to the posterior mean teacher rather than to any individual one (Xu et al. 2025).

Whether the fleet helps depends on which error the teacher makes. Conditioning on the threshold instead of the slip rate splits the grid cleanly -- see the second table above.

Training denoises a teacher who is right and careless. It cannot rescue a teacher who is careful and wrong, and where the standard itself is the error, more of it hurts. A fleet-wide average would report a net gain and hide this entirely.

11. The gate clears every item, and the stream still names the analyst¶

scripts/measure_fleet_linkage.py, src/pharos/fleet.py, results/fleet_linkage.json

Every gate above decides one item at a time. shared_eligible asks whether this entry may train an adapter released at a ceiling; disclosure.decide asks whether this derived label may leave and on whose authority. Both are correct at what they do, and neither is asked the question the design's motivating claim actually turns on: whether a fleet can learn across analysts without leaking what any analyst works on.

A fleet does not disclose an item. It discloses a stream, and a stream carries something no item in it carries -- the set of tasks a contributor was able to weigh in on at all. That set is a function of the contributor's clearance, because an analyst can only decide what an analyst can read.

The attack reads no content: not the report text, not the verdict, not the released label. It sees only which task identifiers appear under a pseudonym, and matches that against what each of the 16 candidate compartment sets would have been able to reach. That is the sharpest available form of the claim, because there is no text in it to redact and no capacity to lower. "Reads no content" is enforced rather than asserted: a test flips every verdict in the stream and requires the attack's output to be unchanged, so a future edit that lets content reach the scorer fails the build.

200 tasks, a fleet of 200 analysts drawn from the full clearance space, every item in the stream already approved by the per-item gates.

Guessing the fleet's most common beat scores 0.105, and the interval excludes it, so the attack is doing real work. But the aggregate is the least interesting number here.

Identifiability is not spread evenly across the fleet. It concentrates on exactly the analysts with the most to protect.

The mechanism is structural rather than statistical. Of the 64 candidate clearances, only 17 produce distinct reachable task sets, and 12 of those 17 belong to exactly one compartment set while a single set is shared by all 16. A cleared analyst can reach the rare, heavily compartmented tasks, and those tasks are diagnostic; an uncleared one reaches only the handful of uncompartmented tasks everyone reaches, and is hidden in the whole candidate space. The pattern holds across five fleet draws (RESTRICTED 0.720 to 0.820, every other level exactly 0.000), so it is not an artifact of one sample.

Nor of one corpus. Across five corpus seeds the reachable-set sizes differ substantially, yet the collapse structure and the recovery rate are identical to three decimals. That is not a coincidence and not a bug: which compartment sets are mutually distinguishable is a property of the label lattice and the channel-to-compartment mapping, not of which events happened to be generated. It becomes so once the corpus is large enough to populate every compartment cell, which the saturation grid puts at 150 events:

Two consequences. The headline runs at 200 events, above saturation, so it measures the lattice rather than a draw. And every small corpus understates the leak rather than overstating it, which is the safe direction but means a short run cannot be quoted as reassurance. It also settles which uncertainty belongs on the number: the interval is over the fleet, because the fleet is the only thing being sampled.

A tie is scored as a failure to identify, never as a hit. That is why three of the four rows are exactly zero: an adversary left choosing among seven candidates has not named anyone, and crediting the tie-break for a coin flip would report a convention as a result.

The controls, each priced in the training volume it destroys.

Bold marks a mitigation that drives recovery to zero. The guessing prior is 0.105, so a row at or below it has closed this attack; read the volume and silenced columns for what that cost. Contributions under the two rulings are 6,606 when compartments are shed and 400 when they are kept.

Three things fall out of that table, and all of them are statements about this attack rather than about the channel.

Textbook k-anonymity at a small k is a no-op, because in a fleet of 200 even a rare clearance cell has several holders: k=10 changes nothing and costs almost nothing. It only bites at k=50, where it takes 79.8% of the training data with it. Choosing the threshold by rank instead of as an absolute count reaches the same protection for a third of that cost (25.5% of the volume against 79.8%), and is the version worth deploying.

Subsampling is dominated on both axes. At p=0.05 it still leaks while destroying 95% of the volume and silencing 108 of 200 analysts. It removes contributions uniformly when the identifying signal is concentrated in the rare ones. (This was our first guess at the mechanism and it was wrong -- we assumed the signal lived in the pattern of absences and would survive suppression of rare tasks. Suppressing the rarest half drives recovery to zero, so it does not.)

Pooling contributors is total and free. Recovery goes to zero, the anonymity set becomes the whole fleet, and not one training example is lost. The reason it costs nothing is structural to the design rather than lucky: personalization is the part that stays local, so the shared side never needed to know who sent what. The cost that remains is the secure-aggregation protocol itself, which is real engineering and is not modelled here.

The tension this creates with finding 10. Finding 10 concluded that the mitigation for learning a wrong standard is annotator-reliability weighting, which requires annotator identity retained through training. This finding says annotator identity is exactly what makes the leak attributable to a person. The two mitigations are in direct conflict, and the conflict is not resolvable by tuning: one wants a per-contributor weight and the other wants no per-contributor anything. Naming it is worth more than picking a side, and the design owes an answer -- most likely a reliability estimate computed under the same secure aggregation that hides the identity, which neither finding measures.

What this means for the design. A disclosure gate that certifies items is not a privacy mechanism for a fleet, and no amount of per-item correctness makes it one. The ruling that makes federation viable at all (finding 2) is the same ruling that opens this channel: shedding compartments takes contributions from 400 to 6,606 and recovery from 0.000 to 0.205. Privacy and utility here are one knob, not two, and it is the knob finding 2 already identified as the design's hinge.

12. Reliability cannot be estimated without identity where it matters¶

scripts/measure_consensus_reliability.py, results/consensus_reliability.json

Finding 10 and finding 11 point in opposite directions, and the conflict is not rhetorical. A learner acquires its teacher's standard exactly, so the mitigation is annotator-reliability weighting, which needs contributor identity retained through training. But contributor identity is what makes a fleet's contributions attributable to a person, and pooling contributors is the only control that removes that leak at no cost in volume. One wants identity; the other wants none.

The obvious reconciliation is to estimate reliability from agreement among contributions on the same task, which needs no identity at all. Several analysts reach the same task, so a contribution that disagrees with its task's consensus could be discounted without anyone knowing who sent it. This measures whether that works.

Nine analysts, 200 tasks, sweeping how many hold a wrong standard. Three aggregation strategies: pool everything; drop contributors whose own agreement with the world is low (needs identity, and is an oracle rather than a method, since it is handed the answer it would have to estimate); or take each task's majority verdict (needs no identity).

It is a cliff, not a curve, and it sits exactly at the majority crossing. Up to 4 of 9, consensus is worth precisely as much as knowing who everyone is. At 5 of 9 it collapses to 0.660, which is not a degraded score but the wrong standard's own agreement with the world -- the bottom row is the same number, because a fleet that is entirely wrong and a fleet that is merely mostly wrong return the same rule. Past the crossing, consensus returns the wrong rule intact and with full confidence.

The fleet sweep shows both properties are structural: across fleets of 5 to 51 the cliff lands at the majority threshold every time and the value it lands on is 0.660 every time.

The canonical estimator falls off the same cliff, to the same value. Dawid and Skene (1979) infer per-contributor error rates and true labels jointly by EM, using no ground truth, and surveys of truth inference report it beating majority voting. Here it matches majority voting exactly at every composition here, collapse included --- at this fleet size, which the sweep above qualifies: from fifteen contributors it gains one step of margin before falling. The mechanism is pinned in tests/test_inference.py: EM started from the majority vote is drawn toward whatever the majority believes, so once most contributors hold the wrong standard it concludes the correct minority are the unreliable ones. It rates them below the wrong majority, inverts the labels, and emits no signal that anything went wrong.

That row matters more than the majority-vote one. An earlier version of this finding compared only against a plain vote, which made "reliability cannot be estimated" a claim about the weakest available estimator. It is now a claim about the strongest one in standard use.

The 9-of-9 row reports the oracle as none rather than 0.000. With every contributor below the floor it drops all of them, and an empty stream is an absence of evidence rather than a perfectly wrong answer. Scoring it 0.000 would say the identity-based control failed, where what it did was correctly refuse everything.

Why this is a negative result and not a reassurance. The identity-free method works exactly in the regime where you do not need it, and fails exactly in the regime where you do. Two findings here say that regime is the realistic one: finding 3b found over-escalation in every model tested, and finding 8 found that agreement is not correctness and that a careful, self-consistent, mutually-agreeing set of reviewers can be uniformly wrong. A fleet of model-assisted analysts is precisely the population in which a wrong standard can hold the majority. There, consensus does not merely fail to help; it certifies the error.

So the conflict between findings 10 and 11 stands. It is not resolvable by pooling, and this measurement closes the cheapest escape from it rather than opening one.

13. A reliability tag can replace identity, and the leak metric cannot tell you when¶

scripts/measure_tagged_aggregation.py, results/tagged_aggregation.json

Three findings box each other in. A learner acquires its teacher's standard exactly, so reliability weighting needs contributor identity (finding 10). Identity is what makes contributions attributable to a person, and pooling is the only free control (finding 11). Consensus cannot recover reliability from pooled outputs (finding 12). Identity is also what an audit trail needs, and auditability is not optional here: the application this testbed serves is specified in terms of auditable training signals, and provenance documentation for high-risk systems became an EU AI Act obligation in August 2026.

The move left is a tag coarser than a person. Label each contribution with the contributor's reliability tier rather than their identity: the aggregator can weight, and nobody is named.

Naming the fleet's most common clearance level scores 0.330, so that is the floor a tag has to beat to be leaking anything.

The two tier rows are indistinguishable on the metric finding 11 uses and opposite on the one that matters. Both name nobody. The independent tag infers clearance at 0.330, exactly the prior, and discloses nothing. The correlated tag infers it at 0.820, because its three groups turn out to be tier0 = 100% OPEN, tier1 = 100% INTERNAL, and tier2 = the two highest levels. It is a clearance label wearing a reliability name, and every contribution carries it.

This is the lower-bound problem made concrete. Finding 11's recovery figures bound leakage from below, and here is a scheme where the bound is vacuous: 0.000 from that attack alongside 0.820 clearance inference from a different question asked of the same stream. Any control validated only against one attack inherits this.

It is also a limitation of finding 11's instrument, and worth stating plainly. That measurement scores an analyst as recovered only when the inference is unique and attributable to them, which is the right question for "was someone named" and the wrong one for "was a group characterised." Any tag coarser than a person scores 0.000 there by construction. Reading that zero as a privacy property would have let this repository's own instrument certify a scheme that discloses clearance on every record.

What this means for the design. The tag threads all three needles, but only under an assumption the mechanism does not enforce and cannot check for itself: that reliability tier is independent of clearance. On a real watch floor that assumption is doubtful, since seniority plausibly drives both how much someone is read into and how reliable their judgement is. So the deployable version of this is not "tag by tier", it is "tag by tier and measure the association between tier and clearance before shipping, and treat a non-trivial association as a disclosure." The measurement is cheap; the assumption is the expensive part.

14. What the agent costs on the hardware it is meant to run on¶

scripts/measure_edge_cost.py, results/edge_cost.json

Every finding above asks whether the triage agent is correct. None asks whether it fits. The application is edge triage on nodes "operating on the scale of a laptop", and a correct agent that does not fit is not a result.

The gap turned out to be half-closed already and never reported. Findings 1 to 3 and 5 were all measured on an 8 GB consumer GPU, which is laptop-class hardware, so those accuracy numbers were edge numbers the whole time. What was missing was a cost attached to them.

Three costs, because they bind at different moments.

A round costs the adapter, not the model. The adapter is 29,933,568 of 3,115,872,256 parameters, 0.961%, so a personalization round puts 57.1 MiB on the wire at bf16 and the base never moves. Doubling that to 114 MiB is the cost of saving a checkpoint in fp32 without noticing, which is worth stating because it is invisible until someone measures the file.

One resident node sustains about 11,200 triage decisions per hour at 0.321 s median. Waking a node costs 3.7 s, or what 11 warm decisions cost, which is the number that matters for a duty cycle: a node that sleeps between watches pays it every time, and one held resident pays it once.

15. The standard privacy mechanism spends the budget on the wrong variable¶

scripts/measure_privacy_budget.py, src/pharos/budget.py, results/privacy_budget.json

Finding 11's control ladder was k-anonymity, rarity suppression, subsampling, and pooling. Every one of those we invented. Differential privacy, the field's default control and the mechanism this work's motivating abstract promises, was simply absent, and stayed absent across three findings, because a self-generated list carries no signal that it is incomplete.

Closing the omission turned out to matter more than adding a row.

Value noise cannot help, at any epsilon. The attack reads the map from pseudonym to task-identifier set. It never inspects a verdict, a released label, or a word of text. Any mechanism acting on contribution values is therefore the identity function as far as the attack is concerned, which is a proposition rather than a measurement. Measured anyway, because a claim that a standard defence is useless should be demonstrated:

At a flip rate of 0.5 the verdict carries no information whatsoever and recovery has not moved. This is the shape of the mistake the finding exists to prevent: noise added to what a system reports while the leak is in what it touches.

Participation noise works, and the composed budget is what it costs. The variable that leaks is whether a contribution about task T exists under pseudonym P at all, so the mechanism is randomized response over that indicator: keep an eligible contribution with probability keep, and fabricate one on an unreachable task with probability fabricate. The second half is what buys deniability and is exactly what subsampling lacks.

Participation noise against a baseline recovery of 0.205 over 200 analysts. epsilon composed is the better of basic and advanced composition across 198 indicators at delta = 1e-05. Read the label-noise column beside it: the setting that suppresses the attack corrupts most of the labels, and an epsilon in the hundreds is a budget in name rather than a guarantee.

Recovery here is again what this attack achieves, so the column bounds leakage from below; the epsilon columns, by contrast, bound it from above for any adversary, which is exactly why a mechanism with a stated budget is worth more than a control validated against one attack. That contrast is the reason this finding is worth having even though its numbers are worse than the ladder's.

The two epsilon columns are the finding. Randomized response bounds the likelihood ratio for one indicator, and the attack observes all of them: two clearances in this fleet are separated by as many as 198 tasks, so the guarantee against an adversary telling one from the other is the budget composed over that set. At the strongest setting tested, a per-indicator ε of 0.41 looks excellent and the composed ε is 66.7, which is not a privacy guarantee in any usual sense. It costs 75.6% label noise to get there.

Quoting the per-indicator figure alone would describe a mechanism far stronger than the one deployed. That is not a hypothetical failure; it is the number a deployment would naturally report.

Two smaller results in the table. Subsampling has infinite epsilon, which is why it sat in finding 11's ladder as a cost with no protection beside it: dropping contributions makes a stream sparser without making any surviving contribution deniable. And recovery at keep=0.9, fabricate=0.1 is higher than the baseline, 0.210 against 0.205, which at 200 analysts is one person and is read as noise rather than as noise-helping-the-attack.

16. The cliff is safe only because the fleets were drawn independently¶

scripts/measure_correlated_fleets.py, results/correlated_fleets.json

Finding 12 swept the number of wrong analysts as a free parameter and found a cliff at the majority crossing. That answers "what happens at 5 of 9" and leaves the question a deployment actually faces untouched: how often does a fleet end up there?

Every fleet measured in this repository has been drawn i.i.d. Nothing enforces that, and it is the most favourable assumption available. Analysts share a training pipeline, inherit a house style, and are corrected by the same supervisors, so a wrong standard propagates through a cohort rather than appearing independently in each person.

Three structures, all with the same expected error rate, differing only in how it is distributed. P(wrong majority) is exact, from a binomial over schools rather than over people. Expected agreement composes that exact probability with agreements measured on real fleets through the real pipeline.

The understatement is largest exactly where a deployment would cite it for reassurance. At a 10% population error rate, independence says one fleet in a thousand crosses the majority; a single shared training culture says one in ten. That is the regime where a designer would say "a wrong majority essentially cannot happen", and it is the regime where the assumption is off by two orders of magnitude. By the time the rates converge, at 0.5, the reassurance was worthless anyway.

Expected agreement follows: 1.000 against 0.964 at a 10% error rate, 0.993 against 0.927 at 20%. Dawid-Skene tracks consensus exactly in every cell, which independently reproduces finding 12's central result on a different fleet distribution.

17. Adding item difficulty does not separate a hard case from a wrong analyst¶

scripts/measure_difficulty_confound.py, src/pharos/inference.py, results/difficulty_confound.json

Finding 12 showed agreement-based estimators failing once a wrong standard holds the majority, and the natural objection is that the estimator was too simple. Dawid-Skene blames the annotator for every disagreement. The obvious missing term is the item: some cases sit near the boundary and everyone struggles with them. Whitehill et al. (NIPS 2009) add exactly that, estimating labeler ability, item difficulty and the true label jointly. That model is now implemented here as pharos.inference.glad.

This corpus can test it unusually well, because it carries a real difficulty structure built for an unrelated reason. The significant class is a conjunction of three facts, and three of the ten background patterns carry two of those three. Those routine items sit one fact from the boundary:

Items whose signature overlap puts them one fact from the boundary are counted in the generated table below, beside the difficulty each fleet estimates for them. Overlap 2 is the near-boundary class: routine under the correct rule, and exactly the class a two-of-three reviewer gets wrong.

The 52 near-boundary items are also exactly the items a reviewer holding a two-of-three standard gets wrong. "This item is ambiguous" and "this reviewer applies the wrong rule" predict identical data.

The control decides it. Estimated difficulty by true overlap, under fleets that differ only in composition:

Estimated item difficulty by true signature overlap, under fleets differing only in composition. Bold in each row marks that row's own peak, read from the artifact rather than asserted in prose.

A correct fleet finds no difficulty structure at all. Spread 1.0, flat across the corpus, because the correct rule resolves a two-of-three item unambiguously: two is not three, so the item is routine and there is nothing to be uncertain about. Every bit of the structure in the rows below it is manufactured by the reviewers.

So the estimator does not fail by being unable to find the wrong analysts. It fails by relabelling a wrong standard as a property of the data, and the two diagnoses call for opposite actions:

• "These items are hard" → clarify the guidance, accept lower accuracy on them, and leave the reviewers alone.
• "A third of the fleet holds the wrong rule" → retrain those reviewers, and expect full accuracy afterwards.

A governance process reading GLAD's output takes the first, which is the wrong one.

And the ability estimate names the wrong people. Difficulty is not the only parameter GLAD reports. It also scores each reviewer, and that score is the one a supervisor would actually query, because it answers whom do I retrain:

Below the majority the estimate is right, and a supervisor acting on it retrains the three reviewers who hold the wrong rule. Above it the ranking flips, and the same supervisor acting on the same field retrains the four who are correct while certifying the wrong standard as the expert one. The ability score tracks the majority rather than the truth, which is finding 12's cliff again in the one parameter that was supposed to survive it: item difficulty was added to explain away disagreement the fleet could not resolve, and it relocates the error rather than isolating it.

This is why the failure is worse than a low agreement number. At 5 of 9, agreement of 0.660 at least announces that something is wrong. The ability column does not: it is confident, well separated, and backwards. The inversion holds at every corpus size swept below, which is more than can be said for the CC-Rasch row.

Random error is not exempt, and has a different signature. The 15%-slip row inflates difficulty with no wrong standard anywhere, but the shape is different in the way that matters: it is elevated across every band, 1.58 to 2.12, rather than concentrated on one. The systematic rows do the opposite --- three of the four bands sit at or below the control value and a single band carries the whole effect.

That is the discriminating shape, and it is not "which band peaks". This section previously said the slip row peaks at overlap 0, furthest from the boundary, and drew the contrast from that. It peaks at overlap 1, and the claim was never measured: it was written beside a run and never re-read against the artifact. The contrast survives the correction because it never depended on which band happened to be highest --- random error raises the floor everywhere, systematic error raises one band and leaves the rest flat. Any annotator error inflates apparent difficulty; only systematic error inflates it at the boundary while leaving the others at the control value.

It is a shape rather than a magnitude: the slip row's spread of 1.343 is smaller than 3-of-9's 2.543, but a single summary number cannot tell you which band carries it, and the peak alone cannot either.

The strongest objection is that GLAD is too coarse, and it does not hold. A two-of-three reviewer is not globally unreliable. They are exactly right on the significant class and wrong only on routine items at the boundary, and GLAD gives each reviewer a single ability number that cannot express that. This is not our observation: Singer et al. make it about GLAD directly, that a single ability per annotator "prevents them from distinguishing majority-class competence from minority-class competence", and their CC-Rasch model conditions both ability and difficulty on the class to fix it. It is five days old at time of writing and is the best available answer to the objection, so it is implemented in pharos.inference.cc_rasch and run on the same fleets:

Read on its own, that bottom row says class-conditioning answers the objection: CC-Rasch recovers the truth exactly where Dawid-Skene and GLAD both collapse to 0.660. It does not, and the reason it does not is the more useful result.

What survives the sweep is the part that does not depend on CC-Rasch. Dawid-Skene and GLAD agree with each other and drop to roughly 0.63-0.67 at the crossing at every size, and GLAD's per-reviewer ability inverts at every size. The confound is established against those two estimators. Whether a correctly identified class-conditional model escapes it is now an open question, and this repository cannot currently answer it.

18. The estimate moves under secure aggregation, and the cliff does not move with it¶

scripts/measure_secure_reliability.py, results/secure_reliability.json

This repository has carried one open problem since finding 13. Finding 10 needs contributor identity to weight a fleet by reliability; finding 11 shows identity is the leak; findings 12 and 13 close both ways of recovering the first from the second after aggregation. The direction never tried was the other one: compute the estimate under aggregation, so the per-analyst stream the attack reads is never produced.

It works, and it is exact rather than approximate, because Dawid-Skene splits along the seam a secure sum offers. Its M step is per contributor over that contributor's own reports, so it runs locally and is never transmitted. Its E step needs a product over contributors per task, which is a sum in logs, and a sum is what secure aggregation reveals and all that it reveals. pharos.secagg supplies masked aggregation over a 64-bit ring and returns a ServerView with no per-client field on it; pharos.inference supplies federated_dawid_skene on top.

The port does not change the answer. Across all ten fleet compositions, the largest posterior disagreement between the federated and centralized estimators is 3.8e-14, with zero label disagreements and matching iteration counts.

And the cliff does not move. It sits at 5 of 9 under both, which was the prediction stated before the run. That localises the problem, and the localisation is the finding: the cliff is not a leak and not an artifact of pooling, it is non-identifiability. Dawid-Skene's parameters are identified only up to a relabelling of the latent class, and the tie-break the literature relies on is diagonal dominance -- FedDS (Dong, Zhu, Shang and Xue, Information Sciences 745:123425, 2026) assumes exactly that in its Eq. (16). A fleet whose majority holds the wrong standard is the fleet where that assumption is false.

What the aggregate discloses anyway¶

Closing the naming channel opens a counting one, and it is worth stating precisely rather than claiming the protocol leaks nothing. Majority-vote initialization is a per-task mean; a mean needs its denominator; that denominator is the number of analysts who could read the task. Nobody is named. But a task's readership is decided by the join of its sources, so reading one headcount is reading, exactly and with no inference, how many analysts are cleared for that join.

Over a 200-analyst fleet, every distinct source-join in the stream yields an exactly correct headcount, with no estimation error at all:

14 of 14 joins yield an exactly correct headcount. Bold marks the exact ones; a row that stopped being exact would lose its bold here rather than in a sentence nobody reran.

The prior column is what makes this a disclosure rather than a restatement of the fleet size: clearances are drawn uniformly from the candidate space, so an adversary who knew only the draw would expect 12.50 analysts on LEGAL+LIAISON and learns the answer is exactly 9. The channel is task-side rather than person-side -- it discloses the fleet's clearance census, not who holds what -- and on a watch floor "how many people are read into LIAISON" is not obviously the safer of the two.

19. An authority of record repairs the cliff, and its price explodes¶

scripts/measure_authority_anchors.py, results/authority_anchors.json

Finding 18 localised the cliff to non-identifiability, and finding 17 already showed no better estimator escapes it from the data alone. What breaks a relabelling degeneracy is an exogenous label: a task whose disposition is asserted by an authority rather than inferred from the fleet. That is the authority of record the build order has owed since step 3, and this prices it.

The scoring rule is the methodology. An anchored task's label was handed over, so counting it would measure how many answers the authority supplied rather than what they bought. Every number below is computed only over unanchored tasks.

Agreement over unanchored tasks only, at anchor seed 909. Bold marks each composition's first budget reaching 0.95; that crossing moves with the draw, which is what the table below reports.

The price is not linear and it is not a curve. It is a second threshold, and it moves far faster than the fleet's error does:

Threshold for 'repaired' is agreement ≥ 0.95 on unanchored tasks, over a corpus of 200. The median is taken over 21 anchor draws with draws that never repair ordered last, so a composition most draws fail to repair reports no number rather than the sweep's upper edge.

At a bare majority an authority ruling on a median of twelve items in two hundred restores the estimate on the 93 that remain scorable. One analyst further and the same repair costs two fifths of the round; two further, three quarters.

The mechanism is visible in the M step: an anchored task constrains every contributor's confusion matrix, but the unanchored majority still outvotes it, so the anchors have to reach a share that dominates the estimate rather than merely inform it.

20. Audit where the fleet splits, and the prediction that said otherwise¶

scripts/measure_audit_policy.py, results/audit_policy.json

Finding 19 drew its anchors uniformly and said so as a limitation: an authority auditing the hardest items would be assuming the question, because knowing which items are hard is knowing where the fleet is wrong. Uniform is an honest floor and a poor proposal. This prices what a selection policy buys on top of it.

A policy may read only what the aggregator can see under finding 18's protocol: per-task vote sums, per-task contributor counts, and the estimator's own posterior. It may not read a per-analyst stream, because there is none, and it may not read ground truth, because then it is an oracle. The oracle is measured anyway, apart from the rest, so that a policy scoring near it is known to be near the ceiling rather than merely better than the floor.

Items an authority must rule on to repair the estimate, out of 97 auditable tasks (200 in the corpus). Lower is better; bold is the winning deployable policy. † oracle reads ground truth and is a bound rather than a method.

uniform is a draw, not a rule: its cell is the median over 21 draws with the full range in brackets. The other policies select from the aggregate and have no draw to vary.

Uncertainty sampling wins, and it ties the oracle exactly. Auditing the tasks the fleet splits on repairs six-of-nine at 20 items against uniform's median 45, and seven-of-nine at 30 against 80. Scoring against the estimator's posterior instead of the raw votes gives the identical answer at every cell.

Half of finding 19's budget was unspendable¶

Only 97 of 200 tasks are ever observed by the aggregator at these compositions. The rest were rejected by every reviewer, so no contributor's confusion matrix touches them and an anchor placed there constrains nothing. Finding 19 drew uniformly from all 200, which means roughly half its budget went to tasks that could not repay it.

Restricting the same uniform draw to the auditable pool is by itself a large improvement --- six-of-nine falls from 100 items to 45 --- before any policy is applied. Read finding 19's "50% of the round" as 50% of the corpus and 47.4% of what was actually auditable -- about half either way.

A fallible authority¶

Finding 19 gave the authority perfect ground truth. Chew and Williams (arXiv:2607.15455, July 2026) build their method on the opposite assumption, that audit labels are themselves noisy and that auditor disagreement reflects genuine ambiguity rather than only random error; they sample the audit set by probability rather than selecting it. Sweeping the authority's own error rate under the winning policy:

The threshold is robust to an imperfect authority. An auditor wrong one time in ten buys the same repair as a perfect one at every composition; only at one error in five does six-of-nine slip from 20 items to 30. That is the reassuring direction, and it is worth stating precisely why: the anchors are exogenous, so what they contribute is information the fleet does not already contain, and a fifth of that being wrong still leaves four fifths pulling against a majority that was uniformly wrong.

21. The corpus the audit policy cannot handle, built on purpose¶

scripts/measure_blind_spot.py, results/blind_spot.json

Finding 20 carried a scope condition: its policy ties the oracle because every wrong standard measured to that point was a shifted escalation threshold, which keys on how much evidence a task shows. Two reviewers differing on that axis disagree exactly on the boundary items, so "the fleet is split" and "the fleet is wrong" were the same set by construction. That is a property of the generator, not of the world. The honest way to find out what the policy is worth is to remove it.

A wrong standard of a different shape. A reviewer who discounts a channel rather than misjudging a quantity: they read every report and decline to credit the ones arriving through one compartment. Nothing about them is noisy or inconsistent. PARTNER is the channel, and the script now refuses to run on a channel too entangled with difficulty --- mean evidence 1.88 on tasks carrying PARTNER against 1.72 on tasks that do not, where SENSOR is 2.00 against 0.48 and is rejected outright.

A fleet-wide PARTNER blind spot changes 20 verdicts of 200, every one on a task showing all three defining facts. The corrupted slice is the unambiguous end of the corpus --- precisely where a threshold-shifted reviewer is always right, and where a fleet is otherwise unanimous and correct.

The advantage does not degrade. It disappears at unanimity.¶

Audited items needed to repair; -- is not reached within 95. Below: the share of a 20-item audit landing on a genuinely corrupted task.

While a sighted minority remains, at seven and eight of nine, the corrupted tasks still draw dissent and margin still finds them: every one of its twenty picks lands on a corrupted task, and it repairs where uniform never does inside the sweep. At nine of nine the signal does not weaken, it ceases to exist --- there is nobody left to disagree --- and margin's hit rate falls to 0.15, next to uniform's 0.10. Chance.

The oracle is unaffected in the only respect that is measured here: it still selects corrupted tasks at 1.00 while every deployable policy falls to chance. That is what makes this a policy failure rather than a hard case --- the corrupted items remain perfectly identifiable to something that can see the answer, and become invisible to everything that cannot. What is gone is the observable, not the tasks.

Whether those tasks are fixable by anchoring is not something this experiment shows, and an earlier version of this paragraph claimed it was. See the retraction above.

22. The trace a blind spot leaves after it stops leaving disagreement¶

scripts/measure_channel_bias.py, results/channel_bias.json

Finding 21 ended badly and left a specific question. A fleet unanimously sharing a channel blind spot is corrupted on a slice of the corpus, every deployable audit policy falls to chance on it, and an oracle on the same data still finds every affected item --- so the tasks are identifiable in principle and invisible in practice. A deployment cannot tell which regime it is in.

That conclusion had a hidden premise. Every signal used to that point was built from disagreement, and unanimity is the absence of disagreement. But disagreement is not the only observable. The fleet's verdict rate is one too, and it can be conditioned on public structure rather than on who said what.

The statistic is a conditional independence test. For an unbiased fleet, whether a task happens to carry a particular channel should say nothing about the verdict once you already know how much evidence the task shows:

with \(V\) the fleet's significant-rate, \(C\) whether the task carries the channel, and \(E\) the count of defining facts visible. A shared blind spot breaks exactly that: at a fixed evidence level, tasks whose evidence arrives through the discounted channel are called routine more often. Conditioning on \(E\) is what makes it a test rather than a correlation --- channels are not spread evenly across difficulty here, and an unconditioned comparison would report an effect for any channel, which is the confound finding 21 had to retract a claim over.

Significance is a within-stratum permutation null, matching the gate's idiom: channel labels are shuffled inside each evidence level, preserving the difficulty distribution and destroying only the association under test.

Verdict noise 0.00

Verdict noise 0.05

Verdict noise 0.15

One-sided permutation p-values against a within-stratum null over 4200 permutations, computed as (b+1)/(m+1) so the floor is 2.4e-04 and never zero. Bold is a detection at p ≤ 0.001, with the stratified gap beside it. ‡ is the channel the fleet actually discounts.

Read the gap for extent and the p-value for detection. The gap is linear in the blind share; the p-value saturates at its floor once an effect is comfortably significant and cannot distinguish shares above that point. p = 1.0000 on a noiseless fleet is not a missing value: with every analyst deterministic, each task's rate is fixed by its evidence stratum, every permutation returns the observed gap, and b = m exactly.

It survives unanimity. At nine of nine blind --- precisely where finding 21's policies score at chance --- the discounted channel sits at the smallest p-value 4200 permutations can produce, at every noise level measured, while no other channel and no healthy fleet is detected anywhere. The statistic reads the level of the fleet's verdict rather than its spread, so the disappearance of dissent costs it nothing. That is the finding, and it holds.

Where the floor actually sits. At \(\alpha = 0.001\), matching the one-sided tail of the three-sigma convention the gate uses elsewhere:

The noiseless row is the one this finding was first reported on, and it is a degenerate fleet rather than a clean one. On a fleet that makes mistakes at the rate this project already treats as realistic, the detector needs roughly a third to a half of the fleet --- still well short of the unanimity that defeats every audit policy in finding 20, which is the comparison the finding exists to make.

Note how narrowly the two borderline rows miss: \(0.0012\) and \(0.0017\) against a threshold of \(0.001\). A less strict \(\alpha\) would move the floor down a row in both cases. The threshold is held at the gate's own convention rather than chosen to make the finding read better, which is the only defensible way to pick one.

What this does not claim. It detects a blind spot aligned with a known, public partition of the corpus --- here the compartment a report arrived through. A shared error that follows no observable partition would leave no trace in this statistic either, and nothing here says how to find one. What has changed is narrower and still worth having: the specific regime finding 21 left as undetectable is detectable, cheaply, from data the aggregator already holds.

23. Once the detector names the channel, provenance finds the corrupted items that are findable at all¶

scripts/measure_blind_spot.py, scripts/measure_audit_policy.py, results/blind_spot.json

Finding 21 ends with every deployable audit policy at chance on a unanimously blind fleet, because all of them read disagreement and unanimity is its absence. Finding 22 supplies the input that was missing: it says which channel is being discounted, from the per-task sums the aggregator already holds. This wires the second into the first.

The policy. Select tasks that carry the named channel, deepest evidence first. It reads two things, both public corpus structure and both already read by finding 22's detector: which tasks carry the channel, and how many defining facts each shows. It never touches ground truth or the per-analyst stream, so it stays inside finding 20's deployability rule.

Evidence depth is not decoration. Carrying the channel is necessary and not sufficient --- the first version of this policy selected on provenance alone and landed 0.50 of its audit on corrupted tasks, against uniform's 0.10. A blind analyst only flips a verdict where the discounted channel was doing the work, which in this corpus is the high-evidence end: the affected slice sits at 3.00 defining facts against a corpus mean of 1.75.

At the unanimity where every other deployable policy is at chance, it ties the oracle. Share of a 20-item audit landing on a corrupted task, at nine of nine blind:

Scope condition, carried with the number every time. The policy ties the oracle because the corrupted slice in this corpus is exactly "carries the channel, and shows deep evidence" by construction --- the same shape of condition finding 20 has to carry. The claim that travels is conditional: when a wrong standard follows an observable partition of the corpus, audit that partition. A shared error that follows no observable partition leaves no trace for finding 22 to detect and no handle for this policy to grip.

24. The crossing is a distribution over corpora, not a share, and "a majority" was nine analysts talking¶

Findings 19 through 23 all describe the same failure in the same words: a majority holds the wrong standard. Every one of them was measured at nine analysts, where the estimator recovers the truth at 4 and collapses at 5. The majority and the crossing are the same cell at that fleet, so the phrase was never wrong. It was never tested either.

Sweeping the fleet and scanning every composition rather than the five the ladder visits separates them.

Every composition from 1 to the fleet size, one EM fit each, over 8 corpus draws, with recovery at agreement ≥ 0.95. The last two columns are counts of draws, not verdicts, and that is the finding: the crossing is a distribution spanning 0.520 – 0.600 rather than a constant, and it becomes less predictable as the fleet grows. At five and nine analysts every draw breaks at the same composition; at fifteen and twenty-five the same fleet survives a bare majority on some corpora and not others.

Agreement past the crossing takes 8 distinct values across draws (0.5773 – 0.7882). Within a single draw it is constant at every composition above the crossing, so the failure still has no gradient — a fleet is identified or it is not — but the level it falls to is a property of the corpus.

The crossing is a distribution, not a constant, and it gets less predictable as the fleet grows. At five and nine analysts every one of eight corpus draws breaks at the same composition. At fifteen and twenty-five they do not agree with each other: the same fleet size survives a bare majority on some corpora and fails on others, 2 draws in 8 at fifteen and 4 in 8 at twenty-five. Across the whole sweep the breaking share spans 0.520 to 0.600.

So the phrase every earlier finding used -- a majority holds the wrong standard -- names the mechanism exactly at nine analysts and is a coin flip at twenty-five. That is the part that survives. What does not survive is the first version of this finding.

This half was already on disk, and nobody read it back into the prose. Finding 17's fleet sweep recorded "Dawid-Skene recovers the truth at a bare majority once the fleet reaches 15" when it ran, and fleet_sensitivity.json has carried dawid_skene_survives_crossing: true for fleets 15, 25 and 51 ever since. Findings 19 through 23 were written after that and went on calling the failure "a majority holds the wrong standard" anyway.

So the new part here is not the fact. It is that the crossing was scanned at every composition rather than at the crossing alone, over eight draws rather than one; that its location is a distribution whose spread grows with the fleet; and the observation that the phrase all five later findings share is load-bearing in none of them. The rest is a claim this repository had already measured and failed to propagate, which is the same prose-outlives-its-evidence failure this page has now corrected five times.

The depth is flat within a draw and not across draws. Past the crossing, agreement is the same value at every composition above it -- checked in all 32 fleet-and-draw cells -- so the failure still has no gradient: a fleet is identified or it is not. But which level it falls to is a property of the corpus, ranging 0.5773 to 0.7882. The earlier claim that the depth is a constant 0.6598 was the same one-draw error as the location.

Reliability weighting buys margin, and less reliably than the first version claimed. Plain consensus flips at the majority by definition, at every fleet size. Dawid-Skene does not, at nine. Above nine it sometimes does and sometimes does not, which is a weaker statement than the one this finding first made. Finding 12 measured this at nine and called it "one extra contributor of margin, not an escape", which remains the right reading of the fleet it had.

The depth does not move where the location does. Agreement past the crossing is 0.6598 at every fleet and every composition above it. There is no gradual degradation between the two levels and no partial failure to find: a fleet is identified or it is not. That is what a relabelling of the latent class looks like, as opposed to a loss of signal, and it is the sharpest evidence in the repository for the non-identifiability reading that findings 18 and 19 argue from.

Everything else in findings 19-23 survives the sweep. Six invariants were checked at four fleet sizes: selection beats a uniform draw wherever a repair is needed, and ties the oracle bound; disagreement collapses to chance at unanimity; provenance recovers every corrupted item there; no policy repairs an unanchored label, including the oracle; and the blinded channel is detected at every noise level while every control stays silent. All six still hold. What moved is confined to this finding's own claims about where the crossing sits and how deep it goes.

A third defect belonged to the sweep itself and is worth recording because it is the failure mode this whole page is about. A smoke run at 300 permutations reported finding 22 as moved. Nothing had moved: a permutation p-value floors at 1/(m+1), so 300 permutations put the entire attainable range above alpha and no cell could be detected at any effect size. The sweep had manufactured a failure to replicate out of its own argument. It now refuses to run below a permutation count that can clear alpha.

25. The cliff is not where we pointed the estimator, except at the crossing itself¶

Every finding from 12 onward rests on Dawid-Skene reporting the wrong standard as the truth once enough of the fleet holds it. And inference.py has said in its own docstring, since the day it was written, that EM started from the majority vote "is drawn toward whatever the majority believes."

That is a researcher degree of freedom sitting directly underneath the headline result, and it is the first thing a reviewer reaches for. The Dawid-Skene log-likelihood is non-convex; majority-vote initialisation carries no global-optimality guarantee. Zhang, Chen, Zhou and Jordan (JMLR 17, 2016) initialise from a spectral method of moments and prove optimal convergence for the two-stage estimator precisely because the conventional start does not have that property; the plainer remedy named in the same literature is many random restarts scored by likelihood. Neither had ever been tried here.

So the question is sharp, and it has opposite answers: is the wrong answer the maximum-likelihood fit, or merely the basin the conventional start falls into? A local optimum is escapable, and findings 12 and 17 through 24 would shrink to statements about one initialiser. A global optimum is not escapable by any initialiser at all.

Every composition the published start gets wrong, over 8 corpus draws and 32 random restarts each, plus an uninformative start, an adversarial one, and the ground truth. An escape is a start that both fits strictly better and recovers the truth.

The gap is signed and the sign is the finding: it is the truth's log-likelihood minus the published answer's, so negative means the wrong answer is the better fit. It is negative everywhere past the crossing and grows with the share, which is why no likelihood-guided initialisation helps there. Exactly zero means the truth is not a fixed point at all — seeded there, EM leaves.

The escape exists, and it is confined to the crossing composition. At the cell where the cliff begins, a better start sometimes recovers the truth: 2 draws in 8 at nine analysts, 1 in 6 at fifteen. Roughly a tenth to a fifth of random restarts find it when it is there. So at exactly one composition per fleet, the published number reports a local optimum rather than the estimator's preference, and that cell now carries the caveat.

Past the crossing, no start escapes, and the artifact says why. The gap between the truth's likelihood and the published answer's runs to tens and then hundreds of nats against the truth. At six of nine it is about 110 nats; at eleven of fifteen, 268. The truth is still reachable there -- seeded at it, EM stays -- but it is the strictly worse fit, so selecting by likelihood rejects it. That is the sharpest form of the result: a better initialiser cannot rescue a search whose objective prefers the wrong answer.

Random restarts are the wrong fix even where an escape exists. Reported as a rate rather than a boolean, because one restart in thirty-two and thirty in thirty-two are the same yes and different methods. Where the escape exists, restarts find it around 11-19% of the time; where it does not, they land on the inverted labelling instead -- agreement near 0.35, which is label switching, the failure the identifiability machinery in the class-conditional model was added to prevent and finding 17's note already records there.

This is the objection that would have been asked first and answered worst. It cost one argument on an existing function and one script; it should have been run before finding 12 was published, not thirteen findings later.

26. Findings 20 to 23 were measured on one corpus, and one of their headlines was that corpus¶

Finding 24 established that the committed corpus seed is favourable: at fleets of fifteen and twenty-five it survives a bare majority where most draws do not. That was measured on the crossing scan alone. The four scripts behind findings 19 through 23 each hard-coded SEED = 7 and did not accept a seed at all, so the corpus was not a sweepable dimension of this project's headline governance results.

It is now. Three defects surfaced before a single policy was scored, all of the same kind as the --fleet defect finding 24 found, and all invisible from the committed artifact.

Finding 29's index picks the rule that wins in 56 of 64 decidable cells across 7 draws. What a wrong call costs is 0.011 to 0.050 of published error rate, and the committed corpus is at the cheap end: quoting its 0.011 as the price of a wrong call would be reporting the draw again. | Blinded channel detected, controls silent | 22 | 8 | 8 |

Fleet of 9, 8 corpus draws, every denominator stated. Finding 21's experiment needs a blind channel orthogonal to item difficulty and refuses to run where they are entangled, so it is constructible on 7 of 8 draws; a draw that cannot host the negative control says nothing about the finding and is excluded rather than counted against it.

The auditable pool an audit budget is a fraction of ranges 83 to 99, not the 97 the script documented. Provenance recovers 0.75 to 1.00 of corrupted items against every disagreement-reading policy's 0.25 or less, so the advantage is robust even where the 1.00 is not.

Finding 19's anchor prices are a median over 21 anchor draws inside one corpus, and the corpus moves them. Across 8 draws, the median number of audited items an authority of record must rule on spans a bare majority 5 to 80, six of nine 80 to 120, seven of nine 120 to 180 (priced in 7 of 8 draws). The committed corpus ranks 2 of 8 at a bare majority, 2 of 8 at six, 2 of 7 at seven when the draws are ordered cheapest first, so it is not a worst case and not a typical one. The published single numbers were one draw's, not the price.

At seven of nine, one draw (seed 101) repairs in only 5 of 21 anchor draws, so it has no median at all: on that corpus an authority usually buys nothing within the ladder. That draw is reported here rather than dropped from the range.

What does not move is the negative. No budget on the ladder repairs unanimity, in any draw -- which is the claim finding 19 is quoted on outside this repository, and the one the open problem is shaped by.

The bound holds below the crossing, and not above it. margin ties the oracle bound in 8 of 8 draws at five and six wrong, and in 6 of 8 at seven. A policy tying the bound means no better selection rule exists on this signal, so the tie is the claim worth having — and it is now bounded by where the fleet is hardest to repair at all, rather than stated everywhere.

Finding 21's row was vacuous, and now is not. disagreement_policies_at_chance read its policy list from a deployable key the blind-spot artifact has never had, so the comparison ran over an empty list and all() returned true without scoring a single policy. The row read 7 of 7 and measured nothing. Scored against the four deployable policies by name, it still reads 7 of 7 — the claim was true, and it had not been tested. Every policy's rate is recorded per draw now, so the prose comparison can be checked against the artifact rather than taken on trust.

What moves next is finding 23's number.

The advantage is robust, and it is the part to quote. Provenance recovers 0.75 to 1.00 of corrupted items in every draw, while every policy that reads disagreement sits at or below 0.25 in every draw. So the comparison finding 23 rests on holds everywhere it was tested; only the absolute value moved. Quote the range and the contrast, never the 1.00.

The negative half is the most robust thing here. No policy repairs an unanchored label holds in 7 of 7 draws, for the channel policy and the oracle alike. The open question finding 23 leaves — whether an audited fleet can be made to generalise from what the authority ruled on — is unaffected by any of this, which is worth stating because it is the twelve-month deliverable.

One more thing that moves, and it is small. The best_deployable policy the artifact names is margin in seven draws and uniform in one. A sweep that reports a single winner from a single corpus is reporting the corpus.

This is the fourth single-draw quantity corrected here, after finding 5's shot count, finding 19's anchor prices, and finding 24's crossing. The class is guarded rather than re-noticed now: every one of these scripts takes --seed, and the test suite fails if a new governance script does not.

The last cell is filled, and it was the most expensive one. Naming measure_authority_anchors as the unswept dimension took an afternoon to close: the script costs 26 seconds a draw, so the exclusion that called it too slow to sweep had been guessing. Finding 19's prices move more than any other quantity here — the bare-majority price spans a median of 5 to 80 audited items where 12 was published, and the committed corpus is the second-cheapest draw of eight at every composition that prices at all. The affordability claim was the optimistic end of a range.

The negative survives, and it is the half that carries the argument: no budget on the ladder repairs unanimity, in any of the eight draws. So an authority of record is costlier than published and still buys nothing where the fleet agrees, which is the shape the open problem already had.

What is pinned is now declared. The multiverse is itself a researcher degree of freedom, and a sweep that picks its dimensions after seeing which are kind to the result is not a robustness check. The artifact carries a multiverse block naming what varies (the corpus seed, eight draws) and what is held fixed with the reason for each: the fleet size, which belongs to finding 24's sweep and would confound this one; the permutation count, because a p-value floors at 1/(m+1); and the 21 anchor draws, which are the inner multiverse this one wraps.

27. Four guards could be inverted with the suite still green, and coverage called them all covered¶

Every finding above rests on code that decides whether a number may be quoted. The question this project asks of everything else had never been asked of that code: if the condition were wrong, would anything say so?

Coverage cannot answer it. validity.py and provenance.py -- the two modules the manuscript names as instruments, a validity flag that travels with the number and the provenance stamp on every artifact -- were already at 100% line and branch coverage. Coverage records that a line ran, and a guard runs identically whether or not anything checks what it decided.

Mutation testing answers it directly: change the condition, and see whether the suite objects. Scoped to those two modules, 438 mutants ran, 187 died and 251 survived. Most survivors are string literals and log-message text, which are noise here. Four are guards, and all four survived the entire suite as it stood, verified by applying each to the real source and running the whole of tests/ rather than the fast subset:

The first is the one worth reading twice. The self-audit already records a case where a sample size was wired to the number of decode regimes rather than the number of tasks compared, and its lesson was that a wrong n produces a plausible number rather than an error. The arithmetic behind n can still be inverted here without a single test failing. The instrument that caught that defect had the defect's own shape inside it.

Seven tests now pin these, and each was verified by re-applying its mutant and watching the suite fail. measure_guard_mutations.py keeps the four as a standing check rather than a one-off: it refuses a dirty tree or a red baseline, applies each mutation to the real source, runs the whole suite, restores in a finally so an interrupt cannot leave a mutated guard behind, and writes results/guard_mutations.json. The artifact records the verdicts now, which is 4 of 4 killed; the table above is what they were before the tests existed. A future change that stops killing one of them is therefore a visible regression rather than a rediscovery. [tool.mutmut] records the wider scope for anyone who wants the full 438: uvx mutmut run.

What is not claimed. 187/438 is not a mutation score for this project. It covers two modules out of thirty, and the survivor count is inflated by string mutants nobody should chase. gate.py belongs in scope and is excluded for cost alone -- its tests take 90 seconds, against half a second for the two modules here -- which is a budget, not a verdict on how well the gate is tested.

28. The open problem, answered: detection converts into coverage, not into correction¶

scripts/measure_selective_risk.py, results/selective_risk.json

Findings 19 to 21 close off correction at unanimity from three directions -- no budget on the anchor ladder repairs a label, no disagreement-reading policy beats chance, and finding 26 shows both hold across corpus draws. Finding 22 then detects the regime anyway, and finding 23 turns that detection into a selection rule that finds the corrupted items. What was left open is the sentence between them: the fleet knows which regime it is in, the items are selectable, and auditing them still corrects nothing. So what should a deployment actually do?

Auditing is not the only action. A deployment that cannot correct a label can decline to publish it. That is selective prediction, and the 2026 form of the rule states it as agreement: predict only when the label is forced, that is when every consistent hypothesis agrees, and abstain otherwise (Khosravani, arXiv:2605.02611). Read across to a fleet of analysts, that rule says publish where the fleet is unanimous. This corpus was built to say what happens to a rule like that when unanimity is the failure.

What is measured. No authority, no anchors, no re-estimation. The estimator runs once per fleet and each policy chooses labels to withhold. Two numbers per cell and neither is readable alone: selective risk, the errors among labels still published, and coverage, how many are published at all. Withholding here is deletion by design rather than by accident -- nothing claims a repair, and a policy that withholds correct labels raises its own risk, which is what makes the column honest without an oracle.

Slip rate 0.0 --- errors among published labels, 20 of the pool withheld

Share of those 20 withheld labels that were actually wrong (chance is the base rate in the column above):

Slip rate 0.15 --- errors among published labels, 20 of the pool withheld

Share of those 20 withheld labels that were actually wrong (chance is the base rate in the column above):

At unanimity the textbook rule is at chance and provenance is exact. Withholding by the channel the detector named halves the wrong labels for 6% of coverage at slip 0.0 and 6% of coverage at slip 0.15, and clears them entirely for 10% of coverage at slip 0.0 and not reached within the sweep at slip 0.15. That is the price of the whole trade: detection converts into coverage, not into correction. Nothing here repairs a label, and the coverage column is what says so.

The same rule on a fleet with no blind spot withholds 10% of labels and catches 0 wrong ones, because there are none to catch. That is what a false detection costs, and it is a number rather than a reassurance.

The channel column is a proposal only where finding 22's detector fires, which is not every row: at 3 of 9 blind it is scored on a channel no deployment would have been told about. Every claim above is quantified over cells where the detector fired, and the artifact lists any that are not.

The control runs the other way. At slip 0.4 a fleet with no blind spot already carries estimator errors, and there the textbook rule works while provenance no longer beats an untargeted draw --- so the failure above is a property of the error's shape, not of abstention, and the remedy is bounded by the same distinction.

The crossing is sharp, and it is at unanimity. With one dissenter left in nine, the textbook rule is exact: margin and posterior withhold nothing but corrupted labels and tie the oracle bound. At nine of nine the same rule lands 3 and 4 of a 20-label withhold on a wrong label, against chance's 2. Nothing about the corpus changed; the signal the rule reads stopped existing. consensus, the textbook inversion, does no better -- at unanimity almost everything is unanimous, so ranking by agreement ranks nothing.

What the answer costs. On the committed corpus, withholding by the channel the detector named clears every label the blind spot corrupted at 10% of coverage, and it needs no authority, no second opinion and no re-estimation. Across the eight corpus draws finding 26 sweeps, seven can host the experiment: the published error rate falls below every untargeted draw in 7 of 7, and the withhold matches what withholding the wrong labels themselves would buy in 5 of 7 -- the same two-in-eight shortfall finding 26 measured for the audit form of this policy, and for the same reason. The qualifier is exact rather than decorative: at a 15% slip rate one error survives the withhold, and it is an independent slip rather than the shared standard -- the rule addresses the component it was aimed at and no other. That is the whole trade and it is worse than an audit would have been if an audit worked: a deployment that detects a shared blind spot buys silence on a slice of its corpus, not a corrected label anywhere. The exchange rate is set by how prevalent the channel is, not by how many labels are wrong -- which is why a false detection is not free either: on a fleet with no blind spot at all, the same rule withholds 10% of labels and catches nothing.

The bound, stated as prominently as the result. All of the above holds where the shared blind spot is the whole of the estimator's error -- the regime the artifact identifies itself, by checking that a fleet with no blind spot has no errors at that slip rate. Push independent noise up until a healthy fleet breaks (slip 0.40 here, where aggregation stops absorbing it) and the picture inverts: the textbook rule works again, because random error does show up as disagreement, and provenance stops beating an untargeted draw, because the shared component is now a minority of what is wrong. Neither rule is right in general. The distinction that travels is between error that is shared and error that is independent, and a deployment cannot pick a rule without knowing which it has -- which is what finding 22's detector is for.

Across draws, in one place. Confidence-based abstention fails at unanimity in 7 of 7 draws, its textbook inversion fails in 7 of 7, provenance works in 7 of 7 and ties the bound in 5 of 7, a false detection costs 10% of coverage in every draw, and the random-error control is measurable and positive in 5 of 7 -- on the other two the estimator is degenerate enough at that noise level that no rule beats an untargeted draw. results/corpus_sensitivity.json carries each denominator.

29. The shape of the error is visible from the aggregate, and it is cheap to read wrong¶

scripts/measure_error_shape.py, results/error_shape.json

Finding 28 ends on a fork it cannot resolve from inside. Withholding by the named channel is right where the error is shared; withholding by confidence is right where it is independent; each is wrong in the other regime. That measurement decided which regime it was in by checking a healthy control fleet's error count -- which an experimenter has and a deployment does not.

The statistic. Condition on the evidence stratum, as finding 22's detector does. Within a stratum every task shows the same evidence, so a fleet applying one rule votes the same way on all of them and the per-task vote sum varies only as its analysts slip. Independent slips are exactly binomial. A shared error is not: it splits the stratum into the tasks the shared standard corrupts and the tasks it does not, and a mixture of two rates carries more variance than a binomial at their mean. So the index of dispersion -- observed variance over binomial variance, pooled across strata -- sits at 1 under independent error and rises above it when part of the error is shared.

It reads per-task vote sums, per-task contributor counts, and the public evidence count. That is strictly less than finding 22's detector needs: no channel to name, no per-analyst stream, no ground truth, no control fleet. Overdispersion diagnostics are standard wherever counts are modelled; what we did not find prior work for is this application, and that is a statement about our search rather than about the literature.

Index of dispersion --- observed variance of the per-task vote sums over the binomial variance at the same rate, within evidence stratum. 1 is independent error; above 1 is a shared component. -- is a fleet with no variance anywhere, which cannot be diagnosed rather than being diagnosed as clean.

Which rule wins, and which the index picks. C is withholding by the named channel, F is withholding by confidence, b is both beating an untargeted draw, - is neither, ? is a fleet the index cannot diagnose.

The index picks the rule that wins in 8 of 10 cells where a rule wins at all. The rate is the less useful half of that sentence: what a wrong call costs is 0.011 of published error rate at worst, because the cells it misses are ones where the winning rule beats an untargeted draw by about one task.

What holds. The index is calibrated on a fleet with no shared blind spot -- it sits at 1 at every slip rate that produces any dispersion at all -- and it rises monotonically with the share of the fleet carrying the blind spot, reaching 9.00 on a noiseless fleet where every analyst is blind. It falls back toward 1 as independent noise rises, which is the mechanism working as described rather than failing: noise fills in the gap between the two groups a shared standard creates.

What does not, and the prediction it refutes. The docstring predicted that thresholding the index would pick the winning rule in most cells and that its failures would land in the ambiguous cells where neither rule dominates. The first half holds and the second is wrong. Both misses are cells where a rule genuinely wins and the index confidently names the other one -- a wrong recommendation rather than an abstention, which is the worse failure mode.

Why the rate is the less useful half. Both missed cells sit at the highest slip rate, where the estimator is already carrying 65 and 76 wrong labels of two hundred, and where the winning rule beats the best untargeted draw by about one task. Following the index there costs 0.011 of published error rate on this corpus. The accuracy rate says 8 of 10; the cost says the two misses are nearly free, and the second number is the one a deployment acts on.

And the cost is a draw, which the sweep caught before this page quoted it as a property. Across the seven corpus draws that host the test the index picks correctly in 56 of 64 decidable cells, and the worst wrong call costs 0.011 to 0.050 -- the committed corpus sitting at the cheap end of that range. An earlier version of this paragraph, and of the manuscript passage it feeds, said "a hundredth of the published error rate" without a denominator. That is the single-draw shape this project has retracted five times, caught here by the sweep rather than by a reader.

The honest form of the answer. The shape of the error is estimable from what the aggregator already holds, and the estimate is worth acting on because its errors are confined to the regime where the choice barely matters -- at a price that ranges to five hundredths of the published error rate across draws rather than the one hundredth this corpus shows. What it cannot do is diagnose a fleet with no disagreement anywhere: with every analyst deterministic and identical there is no variance to compare against, and the artifact reports that cell as undiagnosable rather than as clean. That distinction is the same one this page keeps arriving at from different directions -- a silent instrument and an instrument reporting nothing wrong are different claims.

30. The blind spot with no name is detected the same, and located by the same signal read one-sided¶

scripts/measure_latent_blindspot.py, results/latent_blindspot.json

This repository has carried one residue since finding 23, and its README states it in these words: finding 22 detects a blind spot aligned with a known, public partition of the corpus, and "a shared error that follows no observable partition would leave no trace in it either, and nothing here says how to find one."

That sentence makes two claims and they come apart. The channel scan does lose everything, because a scan enumerates a small public family and an unnameable slice leaves nothing to enumerate. The index of dispersion never read a partition in the first place --- it reads per-task vote sums within an evidence stratum --- and the question is whether it notices the difference. It does not.

The construction, matched to finding 21's in every respect but one. AnalystPolicy.distrusted_reports is the same wrong standard as blind_compartment with its handle removed: the blind analysts decline to credit a set of individual reports rather than a channel. The set is drawn so it corrupts what PARTNER corrupts on this corpus --- 20 verdicts of 200, all on tasks showing all three defining facts, so a lost report drops the reviewer below a conjunction of three --- and drawn uniformly within that stratum, which is what makes membership unpredictable from any column the corpus records. A balance precondition refuses a draw whose compartment carriage sits above the 99^th percentile of uniform draws from the same pool, because a slice that did load onto a channel would be found by the scan and this finding would report the scan working as the scan failing.

What each detector says, by what the blind spot is keyed on. scan is the channel detector of finding 22, which enumerates the corpus's compartments. index is the dispersion statistic of finding 29, which enumerates nothing. The latent columns are the median and range over 5 slice draws; -- is a fleet with no variance to read. The unblinded row is marked ref: with no blind analyst the two constructions build the same fleet, so it cannot disagree and is excluded from the verdict, which rests on the 10 rows that can.

Localization at unanimity, where every disagreement-reading rule is already at chance. Share of a 20-item withhold that landed on a label the estimator got wrong, over every slice draw and slip rate. uniform is the best of 21 untargeted draws rather than their median, because a median is not a floor and finding 28 had to retract a claim that beat one. oracle reads ground truth and is a bound rather than a method.

As the corrupted slice grows past the majority of its stratum. The pool of tasks a discounted report can flip is 69 on this corpus, so the crossing sits between the two middle rows. Every cell is the median over 5 slice draws with the range beside it, because a slice is a sample and this sweep quoted one draw of it as a constant until it did not.

Detection survives the loss of the partition, exactly. At every share of a deterministic fleet the two constructions give the same index --- 0.73, 2.36, 4.23, 6.42, 9.00 --- and the reason is worth stating because it is also the limit of the statistic: within a stratum the index depends on how many tasks the shared standard corrupts and not on which ones, so two slices of the same size in the same stratum are the same number to it. Once analysts slip independently the two fleets are different draws and the indices separate by a tenth at worst and hundredths typically --- 0.13 in the widest cell, under 0.10 in every other; the channel-keyed value then sits inside the range five latent slice draws produce, in every cell that could fail. The statistic cannot tell the two constructions apart.

The scan is the mirror image. It is silent on the latent construction in every cell, and it names PARTNER on the channel-keyed twin wherever it can name anything --- which is not everywhere, and the table says so: at a 15% slip rate it does not fire below five of nine, which is finding 22's own detection floor rather than anything this construction did. The rows where both columns read silent are the rows where the instrument has no power, and reading them as agreement would be the mistake the paragraph above exists to prevent.

Localization survives too, and that was predicted to fail. The docstring's third prediction was that ranking tasks by their within-stratum residual --- the per-task summands of the index --- would do no better than an untargeted draw at unanimity, on finding 21's reasoning that a unanimously blind fleet leaves nothing to distinguish a corrupted task from a merely unusual one. It is wrong. At unanimity deviation lands a median 1.00 of a 20-item withhold on a wrong label against the best of 21 untargeted draws at 0.25, tying the oracle, with no channel named and no detection required first. The three disagreement-reading rules are all below that untargeted floor on the same fleets --- margin and posterior at 0.12, consensus at 0.10 --- which is finding 28's result arriving again from a fleet with no channel in it.

The reconciliation is the sentence above: the index is invariant to which tasks a standard corrupts, and its summands are not. A sum over residuals discards the information a rank over the same residuals keeps, and finding 29 read only the sum.

What the two-sided form costs, measured rather than asserted. The fourth prediction was that if localization worked at all it would work only while the corrupted slice was a minority of its stratum, and the sweep was run to find the crossing. It is there. Once the corrupted slice sets its own stratum's rate the clean tasks become the outliers, and deviation falls to or below an untargeted draw in 5 of the 12 swept cells, none of them below 40 of the 69 eligible tasks --- the pool's majority is 35, and the first inversion is on the other side of it. The worst cell is 40 of 69 on a noiseless fleet, where it lands 0.00 of a 20-item withhold on a wrong label, in all five slice draws, against an untargeted 0.35. That is the failure mode that matters, because a rule that is merely uninformative wastes a budget and a rule that is anti-informative spends it on correct labels. Degradation starts earlier than inversion: at 30 of 69 with analysts slipping it is already at a median 0.65 against a perfect 1.00 one row above.

Every cell in that sweep is a median over the same five slice draws the grid uses, with its range printed beside it. It was measured at one draw first, and the difference is instructive rather than cosmetic: the structural claims are identical across draws --- the same inverted sizes, the same tie count --- while individual cells move by up to a tenth, so a single draw would have published 0.10 and 0.75 as though they were constants. This page has retracted five numbers that were properties of one sample.

Signing the residual removes the crossing, and names the assumption that does it. shortfall is the same statistic read one-sided: audit where a stratum votes low, not where it votes far. A blind spot is a failure to credit evidence and a reviewer who credits less escalates less, so the corrupted tasks vote low whether they are ten of a stratum or fifty. It never falls to an untargeted draw in any cell of the sweep, and it ties the oracle in 10 of the 12. Both misses are worth naming rather than rounding away: at 10 of 69 with slip it takes 0.50 against the bound's 0.55, one task short, and at 60 of 69 with slip it takes a median 0.50 against 1.00, which is the real limit and is the subject of the paragraph below.

That directional assumption is the scope condition and it is not new here: finding 22's detector is already one-sided, for this reason and in these terms. A shared error that raised the rate --- a fleet crediting something it should not --- needs the mirror of this rule, and nothing in the aggregate says which of the two a fleet has. The honest statement of what a deployment gets is therefore conditional in one place rather than two: it no longer needs a nameable channel, and it still needs to know which way its analysts are wrong.

Where it does break. At 60 of 69 corrupted, with analysts slipping at 0.15, shortfall falls to a median 0.50 against the oracle's 1.00 --- nine of the sixty-nine remain clean, and a rate estimated from a stratum that is seven-eighths corrupted is no longer a reference. That is the same shape as every other limit on this page: the statistic needs a majority of the stratum to be doing the right thing, which is a weaker requirement than the majority of the fleet that findings 19 to 21 needed, and is a different quantity rather than a smaller one.