Google DeepMind has published the methodology and results for evaluating Gemini 3 Pro, assessing its performance across reasoning, multimodal capabilities, agentic tool use, multi-lingual tasks, and long-context windows. The evaluation utilizes pass@1 scoring with single-attempt settings via the Gemini API, comparing the model against competitors like Claude Sonnet 4.5 and GPT-5.1 using self-reported or official leaderboard data where available.
- Reasoning benchmarks include Humanity's Last Exam, ARC-AGI-2, and MathArena Apex, with results sourced from ScaleAI, the ARC Prize website, or matharena.ai.
- Image capabilities were tested on MMMU-Pro, ScreenSpotPro, CharXiv Reasoning, and OmniDocBench 1.5, utilizing specific API parameters like media_resolution for high-fidelity inputs.
- Video performance was measured using Video-MMMU with a recommended setting of 280 tokens per frame and temperature 0.
- Code evaluation covered LiveCodeBench Pro (ELO rating), Terminal-Bench 2.0, and SWE-bench Verified, employing single-attempt scaffolding for the latter.
- Tool use was assessed via τ2-bench across Retail, Airline, and Telecom categories, alongside Vending-bench 2 results from Andon Labs.
- Factuality and long-context capabilities were measured using the FACTS Benchmark Suite, SimpleQA Verified, and MRCR v2 for 128k and 1M context windows.
The article states that Gemini 3 Pro significantly outperforms Gemini 2.5 Pro across the evaluated benchmarks as of November 2025.