Skip to content

The seed effect at the half size is above 0.01, so the decomposition runs at five seeds

The registered seed rule says three seeds up to a seed main effect of 0.01 and five above it. It was measured at 0.000 on the full organizations and 0.013 at about 1,800 examples; the re-registration trains on halves, so the rule has to be read at the half size. The three-seed placebo runs already are that measurement: OpenStack's halves trained on 2,157 and 2,159 examples, at the size every half-adapter will train on, and Qt's on 4,205 and 4,206. Read with scripts/seed_effect.py over each placebo's three single-seed runs.

placebo sigma_b one-sided 95% upper bound artifact
OpenStack halves 0.0106 0.086 seed-effect-placebo-openstack.json
Qt halves 0.0148 0.084 seed-effect-placebo-qt.json

Both point estimates are above 0.01, so five seeds, by the rule fixed before either run. The upper bounds are wide because two contrasts of three seeds each give two degrees of freedom, which is itself the reason not to rest a three-seed decision on them. This is the case the spec named: three seeds only if the effect at the half size is bounded below 0.01, and it is not.