A guide to "loop engineering" explains how AI agents can perform autonomous machine learning research by replacing manual iteration with automated loops. The article details Andrej Karpathy's open-source repository `autoresearch` and the `Bilevel Autoresearch` paper as verified artifacts for this pattern.
- Karpathy released `autoresearch` on March 7, 2026, an MIT-licensed tool that allows agents to edit training code while keeping evaluation utilities fixed, preventing self-deception.
- In tests on GPT-2-quality code, the loop ran approximately 700 experiments over two days, identifying 20 genuine improvements that cut training time by 11%.
- Shopify CEO Tobi Lütke reported a 19% improvement after 37 overnight experiments using the same tool on an internal model.
- The `Bilevel Autoresearch` paper introduces an outer loop that modifies the inner loop's search mechanisms, achieving a validation bits per byte drop five times larger than the single loop alone.
These loops allow AI to autonomously refine models or code by continuously proposing changes, verifying results against strict metrics, and persisting only valid improvements.