Researchers introduce s1, a method for achieving test-time scaling in language models through a straightforward approach involving curated data and compute control. The team created the s1K dataset of 1,000 questions with reasoning traces and developed "budget forcing" to manage extra test-time compute by extending or terminating the model's thinking process.
- Supervised finetuning of Qwen2.5-32B-Instruct on s1K combined with budget forcing allows the model to double-check answers, often correcting incorrect reasoning steps.
- The resulting s1-32B model exceeds OpenAI's o1-preview by up to 27% on competition math questions from MATH and AIME24 benchmarks.
- Scaling s1-32B with budget forcing enables performance extrapolation beyond its baseline, improving scores on AIME24 from 50% to 57%.
This work demonstrates that simple test-time scaling techniques can replicate and surpass the reasoning capabilities of more complex proprietary models like o1-preview.