The authors introduce a pre-registered screening rule that determines before implementation whether an evolutionary outer loop over neural network parameters is worth building compared to a cheap single-shot alternative. The rule calculates a recovery metric R, defined as the best single-shot gain divided by the best gain of any cheap method, and prescribes skipping the outer loop when R is greater than or equal to 90%.
- The screening rule computes a single number at a Phase-0 gate: R = s/G, where s is the best single-shot gradient/curvature statistic's gain and G is the best gain of any cheap method evaluated.
- Validation on two analyzed cases showed the gate fired with R approximately 1.0 in both instances, leading to the abandonment of the outer loops.
- In one case, a companion factorial decomposition localized the apparent win to a static substrate change, showing the evolutionary lifecycle contributed no detectable gain.
- On one project, the gate cost about 50-70 GPU-hours and screened out an estimated 400+ GPU-hours plus weeks of implementation, resulting in a 6-8x saving.
The rule is prospectively falsifiable, as a task with R less than 90% where the outer loop fails to beat single-shot would refute it.