The seed effect at the half size is above 0.01, so the decomposition runs at five seeds¶
The registered seed rule says three seeds up to a seed main effect of 0.01 and five above it. It
was measured at 0.000 on the full organizations and 0.013 at about 1,800 examples; the
re-registration trains on halves, so the rule has to be read at the half size. The three-seed
placebo runs already are that measurement: OpenStack's halves trained on 2,157 and 2,159 examples,
at the size every half-adapter will train on, and Qt's on 4,205 and 4,206. Read with
scripts/seed_effect.py over each placebo's three single-seed runs.
| placebo | sigma_b | one-sided 95% upper bound | artifact |
|---|---|---|---|
| OpenStack halves | 0.0106 | 0.086 | seed-effect-placebo-openstack.json |
| Qt halves | 0.0148 | 0.084 | seed-effect-placebo-qt.json |
Both point estimates are above 0.01, so five seeds, by the rule fixed before either run. The upper bounds are wide because two contrasts of three seeds each give two degrees of freedom, which is itself the reason not to rest a three-seed decision on them. This is the case the spec named: three seeds only if the effect at the half size is bounded below 0.01, and it is not.