A systematic study evaluates how well coding agents can patch compiler missed optimizations, identifying that success requires generalizing beyond the specific reported case to similar instances.
- The researchers constructed a benchmark using real-world LLVM missed optimization issues to compare agent-generated patches against those from human developers.
- Results indicate that while agents often optimize the provided examples, their patches frequently cover only part of the intended scope or partially overlap with it.
- In some cases, agent-generated patches generalize beyond the reference patch provided by developers.
- The study introduces historical-knowledge augmentation techniques leveraging prior LLVM optimization pull requests through retrieval and distillation.
- These augmentation techniques improve developer-aligned generalization and yield practical benefits when applied to real-world IR.
The findings highlight the complexity of automating compiler optimizations, suggesting that simple example fixing is insufficient without broader generalization capabilities.