media Medium (Barnacle Goose) · 865d ago

LMSYS benchmark ranks Claude 3 Opus top with Elo rating of 1253

from English

Researchers from LMSYS have released a leaderboard using an Elo rating system to compare large language models, revealing that Anthropic’s Claude 3 Opus has secured the top spot. The model achieved an Elo rating of 1253 based on head-to-head battles evaluated by human voters.

Claude 3 Opus holds the highest rating with 33,250 total votes and a +5/-5 record.
OpenAI’s GPT-4 preview models follow closely with ratings of 1251 and 1248.
Google’s Bard (Gemini Pro) ranks fourth with an Elo rating of 1203.
The older GPT-4–0314 model is ranked sixth with a rating of 1185.

This benchmark provides an objective method to track the rapid advancements in language AI and identify leading architectures as competition intensifies among major tech companies.

Importance 2/3 Medium (Barnacle Goose) Anthropic OpenAI Benchmark results

Benchmarks

Benchmark	Model	Score
LMSYS Arena (Elo)	Claude 3 Opus	1253Elo
LMSYS Arena (Elo)	GPT-4 preview models	1251Elo
LMSYS Arena (Elo)	Bard (Gemini Pro)	1203Elo
LMSYS Arena (Elo)	Claude 3 Sonnet	1198Elo
LMSYS Arena (Elo)	GPT-4–0314	1185Elo
LMSYS Arena (Elo)	Mistral-Large-2402	1157Elo
LMSYS Arena (Elo)	Qwen1.5–72B-Chat	1148Elo

Read original

media MarkTechPost · 23d ago

Mistral Vibe for Code leads four coding agents on scaffold-to-PR task

A capability comparison of Mistral Vibe for Code, Claude Code, OpenAI Codex, and Cursor evaluated their performance on a multi-stage workflow involving scaffolding, testing, and pull request generation. Scores were derived from documented features, published benchmarks, and vendor specifications as of July 14, 2026.

media Latent Space · 26d ago · 7 views

OpenAI Codex reaches 7M users, up >10x in 6 months

OpenAI's Codex product has reached 7 million users, representing more than a 10x increase from approximately 550k-700k users at the start of the year. This rapid growth coincides with the July 9 launch of GPT-5.6 Sol and recent infrastructure updates to manage usage.

arxiv arXiv cs.CL · 32d ago · 4 views

RuBench 1.0 introduces Russian-language repository-level agentic coding benchmark

The authors introduce RuBench 1.0, a benchmark of 25 tasks mined from recent fix commits in five live open-source repositories (aiohttp, aiogram, Laravel, NestJS, Fastify). Each task is specified natively in Russian and judged by upstream maintainer regression tests withheld from release.

arxiv arXiv cs.CL · 35d ago

Zero-shot evaluation shows Gemini leads LLMs on 13-class emotion taxonomy

A study evaluated three commercial large language models—Claude (claude-sonnet-4-6), ChatGPT (GPT-5.4), and Gemini (gemini-2.5-flash)—on a zero-shot fine-grained emotion classification task using a stratified 1,000-sentence sample from the boltuix/emotions dataset.

arxiv arXiv cs.CL · 51d ago · 1 view

REDACT: Multilingual PII Benchmark with Systematic Control

REDACT introduces a systematically controlled multilingual benchmark for personally identifiable information detection, featuring 51 entity types, 4,127 surface-form patterns, and 25 languages. It evaluates five detectors across 1,000 records, revealing that rule-based models fail on high-stakes data while LLMs perform better, especially in high-sensitivity categories. A reference-free LLM assessment confirms sensitivity-tier assignment as the most challenging evaluation axis.