IBM open-sources CodeAlchemy, a massive synthetic dataset of high-quality code

IBM has open-sourced CodeAlchemy, a pipeline and synthetic dataset designed to improve AI model performance by pairing code with execution traces. The release includes nearly 1 trillion tokens across 15 programming languages, totaling at least 200 times the size of Wikipedia.

The dataset contains 1.3 million code files paired with actual execution traces generated in sandboxed environments.
It covers 15 languages including Python, C++, Java, and Rust, surpassing previous datasets like Nemotron in scale.
Researchers created new benchmarks, DevEval and TraceEval, to test model intent inference and runtime simulation.
Training on CodeAlchemy improved the Granite 4.0 3B model's win rate against Claude Sonnet from 2% to 8% on DevEval.

The synthetic data aims to teach models how code behaves at runtime, addressing a key limitation of static code training and enabling better performance for smaller, more efficient models.

Benchmark	Model	Score
HumanEval	Granite 4.0 3B	83.5%
MBPP	Granite 4.0 3B	63.2%

Benchmarks