DecompRL learns modular code generation to cut GPU costs by 50x

DecompRL is a reinforcement learning algorithm designed to help Large Language Models solve complex coding problems by decomposing them into smaller, independently solvable sub-functions. This approach shifts the computational bottleneck from expensive GPU inference to cheap CPU evaluation.

• The method explicitly learns to generate hierarchical code structures that can be recombined to produce candidate solutions.
• Recombining implementations of modules yields up to $k^{n}$ candidates, reducing GPU token costs by approximately 50x.
• On LiveCodeBench and CodeContests using Qwen 2.5 7B and Code World Model 32B, DecompRL outperforms standard RL baselines beyond 10^5 tokens per problem.

By making the task easier through decomposition rather than relying on repeated sampling, the algorithm enables models to solve problems that standard generation methods cannot reach.