Google has released an experimental model, Gemini-Exp-1114, which now ranks joint No. 1 on the Chatbot Arena leaderboard alongside OpenAI's ChatGPT-4o-latest. Tested with over 6,000 community votes, the model achieved an overall Arena score of 1344, surpassing previous Gemini versions and beating OpenAI's o1-preview.
Gemini-Exp-1114 ranks No. 1 on the Vision leaderboard and in two unspecified categories, while placing No. 3 in coding and Hard Prompts with Style Control. In direct comparisons, it wins 50% of votes against GPT-4o-latest, 56% against o1-preview, and 62% against Claude-3.5-Sonnet.
Developers can currently test the model via Google AI Studio, with API access expected soon.