Skip to content

Power at a test window's realised size implemented

"Fetch horizon" registers that a test window holding fewer changes than its simulation projected is reported beside its verdict with the simulation's power at the realised size. scripts/power_at_size.py computes it. It takes the test report and the simulation the report was read under, and refuses a simulation whose organization, K, projected size, calibration or bounds differ from the report's, a report at or above its projection, and a corpus whose rebuild does not give the simulation's pilot changes and partitions. It reruns the simulation's trials, at its seed, at the number of changes the cell was read on, at the lift that registered each level's bound and at the calibrated run shift. Neither the lift nor the shift depends on the test size, so no calibration or bisection is redone. It records the share of trials supported at each level and its Monte Carlo standard error, sqrt(p(1 - p) / trials), as Morris, White and Crowther (Stat. Med. 2019) ask of simulated performance. Power is at the registered bound, never at an observed effect. Both gates take these artifacts as powers, require one for every organization in below_projection and none for any other, check each Monte Carlo SE against its power and trial count, and report each level's power in the cell's size. partition_sensitivity.py now records its seed, and the pool and partition rebuild both scripts use is one function (pools).