media NextBigFuture · 358d ago

xAI Grok 4 leads Arc-AGI2 benchmark with 15.9% score versus GPT5

from English

xAI's Grok 4 model achieves a 15.9% score on the Arc-AGI2 benchmark, outperforming OpenAI's GPT5 which scored 9.9%.

Grok 4 holds a significant lead over GPT5 on the Arc-AGI2 benchmark.
xAI plans to release Grok 5 by the end of 2025.
Grok 5 training will utilize compute equivalent to approximately 3 million Nvidia H100s or 550,000 B200 chips.

Importance 2/3 NextBigFuture xAI OpenAI Benchmark results

Benchmarks

Benchmark	Model	Score
ARC-AGI 2	Grok 4	15.9%

Read original

media r/LocalLLaMA · 2h ago

DeepSeek V4 Flash GA ranks same as Sonnet 5 and Grok 4.5 on DeepSWE

DeepSeek claims that its newly generally available model, DeepSeek V4 Flash, achieves performance parity with Anthropic's Sonnet 5 and xAI's Grok 4.5 on the DeepSWE benchmark.

media Jeremy Berman (Substack) · 292d ago

GPT-5 Pro solves new FrontierMath Tier 4 problem, edging Gemini 2.5 Deep Think

OpenAI's GPT-5 Pro set a new record on the FrontierMath Tier 4 benchmark by solving one additional research-level math problem compared to Google's Gemini 2.5 Deep Think. The evaluation involved manually grading three compute-intensive model settings on 50 extremely hard problems developed by professional mathematicians.

lab OpenAI News · 2d ago · 6 views

Two API settings tripled GPT-5.6 scores on ARC-AGI-3

Enabling two specific API settings for GPT-5.6 resulted in a tripling of its performance scores on the ARC-AGI-3 benchmark.

media Together AI Blog · 5d ago · 1 view

Kimi K3 beats GPT-5.6 Sol on DeepSWE pass@4 and costs 64% less

A comparison of Kimi K3 and GPT-5.6 Sol on the DeepSWE benchmark reveals that while GPT-5.6 Sol leads in single-shot quality (72.7% vs 68.5%), Kimi K3 achieves higher pass@k scores for k > 1 at a significantly lower cost.

arxiv arXiv cs.CL · 11d ago · 1 view

Intern-BioBreaker reveals physical biosecurity risks in frontier LLMs

Researchers developed Intern-BioBreaker, a specialized bio-red-teaming model, to assess the biological risks of frontier large language models by coupling computational stress testing with wet-lab validation. The study found that aligned models can be induced to provide operational guidance for safety-sensitive tasks and generate sequence-level outputs with harmful properties.