media 1AI.net · 816d ago

OpenAI secretly tested GPT-4o in Chatbot Arena, topping the leaderboard

from English

OpenAI employee William Fedus confirmed that the mysterious chatbot "gpt-chatbot" appearing on the Chatbot Arena leaderboard is a pre-release version of the newly launched GPT-4o model. The model achieved the highest score ever recorded on the platform during these secret tests.

OpenAI tested multiple versions of GPT-4o in the arena starting in April, using aliases such as "gpt2-chatbot" and "im-also-a-good-gpt2-chatbot".
Upon release, GPT-4o surpassed previous top models Claude 3 Opus and GPT-4 Turbo by a significant margin of approximately 50 Elo points.
As of press time, the test model held an Elo score of 1309, compared to 1253 for GPT-4-Turbo-2023-04-09 and 1246 for Claude 3 Opus.

The official LMSYS account noted that the public version of GPT-4o has now entered the arena and will soon appear on the public leaderboard.

Importance 3/3 Beats a top-lab benchmark 1AI.net OpenAI Benchmark results

Benchmarks

Benchmark	Model	Score
LMSYS Arena (Elo)	GPT-4o	1309Elo

Read original

media Together AI Blog · 2d ago · 7 views

DeepSeek-V4 Flash 0731 vs GPT-5.6 Luna on DeepSWE: Cost and Coding

A comparison of DeepSeek-V4 Flash 0731 and GPT-5.6 Luna on the DeepSWE benchmark reveals that while GPT-5.6 Luna is the stronger engineer, a cascade strategy using both models achieves higher accuracy at lower cost.

media r/LocalLLaMA · 9d ago · 1 view

DeepSeek V4 Flash GA ranks same as Sonnet 5 and Grok 4.5 on DeepSWE

DeepSeek claims that its newly generally available model, DeepSeek V4 Flash, achieves performance parity with Anthropic's Sonnet 5 and xAI's Grok 4.5 on the DeepSWE benchmark.

lab OpenAI News · 10d ago · 6 views

Two API settings tripled GPT-5.6 scores on ARC-AGI-3

Enabling two specific API settings for GPT-5.6 resulted in a tripling of its performance scores on the ARC-AGI-3 benchmark.

media Together AI Blog · 13d ago · 1 view

Kimi K3 beats GPT-5.6 Sol on DeepSWE pass@4 and costs 64% less

A comparison of Kimi K3 and GPT-5.6 Sol on the DeepSWE benchmark reveals that while GPT-5.6 Sol leads in single-shot quality (72.7% vs 68.5%), Kimi K3 achieves higher pass@k scores for k > 1 at a significantly lower cost.

arxiv arXiv cs.CL · 19d ago · 2 views

Intern-BioBreaker reveals physical biosecurity risks in frontier LLMs

Researchers developed Intern-BioBreaker, a specialized bio-red-teaming model, to assess the biological risks of frontier large language models by coupling computational stress testing with wet-lab validation. The study found that aligned models can be induced to provide operational guidance for safety-sensitive tasks and generate sequence-level outputs with harmful properties.