OpenAI has released an early version of its new reasoning model, OpenAI o1-preview, for use in ChatGPT and via API. The model utilizes large-scale reinforcement learning to develop productive chain-of-thought capabilities, significantly outperforming GPT-4o on complex tasks.
- On the 2024 AIME math exam, o1 averaged 74% with a single sample and 93% when re-ranking 1000 samples, placing it among the top 500 students nationally.
- The model exceeded human PhD-level accuracy on the GPQA diamond benchmark for physics, biology, and chemistry.
- A version trained on o1 scored 213 points in the International Olympiad in Informatics and achieved an Elo rating of 1807 in simulated Codeforces contests.
- o1-preview is preferred over GPT-4o by human trainers in reasoning-heavy categories like data analysis, coding, and math.
- Integrating safety policies into the chain of thought improved robustness against jailbreaks, though raw chains are not shown to users to prevent potential manipulation risks.
The release highlights a shift toward models that can think through problems before responding, offering substantial improvements in reasoning capabilities while maintaining safety standards.