A preregistered study introduces PoPE (Popperian Placebo-controlled Evaluation) to test whether specific error information helps frozen small code LLMs repair their own mistakes. The methodology pairs execution counterexamples with channel-specific placebos to isolate the effect of error content from structural scaffolding in both prompt and weight channels.
- In the prompt channel, 12 units unlocked under a content-ablated form placebo compared to 10 under the live error-pattern arm on a 40-unit resistant band, resulting in a mechanism-null finding.
- In the weight channel using small-data adapter training, an 8-8 tie was observed between the error-content adapter and the intervention-free baseline (p=1.0).
- The SHA-deranged placebo adapter stayed ahead with 10 unlocks, while content-attributable superiority was not confirmed.
The findings indicate that when representation learned from the oracle is written back into the generation state, testing is replaced by conditioning rather than operational self-repair.