media Latent Space · 1h ago · agents

Gemini 3.7 Flash update brings GDM back to the forefront

from English

The article highlights a chart from the Gemini 3.7 Flash update showing that previous versions, specifically 3.5 and 3.6 Flash, had fallen behind more recent Claude 4.8+ and GPT 5.5+ series models.

The update aims to address performance gaps against competitors like Claude 4.8+ and GPT 5.5+.
A chart is cited as the most compelling aspect of the announcement, illustrating the previous lag in performance.

Importance 2/3 Latent Space Google DeepMind Benchmark results

Read original

arxiv arXiv cs.AI · 1d ago · 2 views

VITA clinical RAG matches or outperforms frontier LLMs on HealthBench

A purpose-built retrieval-augmented generation system named VITA, designed for low- and middle-income settings, was evaluated against general-purpose large language models on the HealthBench benchmark. The study demonstrates that corpus-specific design can maintain competitive performance even as newer frontier models are released.

arxiv arXiv cs.CL · 1d ago · 1 view

Corpus-specific VITA RAG matches or beats frontier LLMs on HealthBench

VITA, a retrieval-augmented generation system designed for low- and middle-income settings, ranks first on the English-language subset of HealthBench, outperforming GPT-5.4, o4-mini, Gemini 3.1 Pro, and Claude Sonnet 4.6.

arxiv arXiv cs.AI · 39d ago

TestEvo-Bench: Executable live benchmark for test and code co-evolution

Researchers introduce TestEvo-Bench, a live benchmark designed to evaluate how well agents handle the co-evolution of software tests and code. Unlike existing benchmarks that rely on static metadata, this framework uses real commit histories and environment configurations to verify executable metrics like pass rate and coverage.

media r/LocalLLaMA · 39d ago

Meta tested ChatGPT, Gemini, and Character.AI with minor-perspective crisis prompts

Meta reportedly had hundreds of contractors pose as minors and send suicide, sex, and drug-related prompts to chatbots from OpenAI, Google, and Character.AI. In a single testing round, more than 45,000 prompts were sent.

arxiv arXiv cs.CL · 39d ago · 1 view

QLoRA fine-tuning of Gemma 4 31B closes cross-lingual relation extraction gap for Romanian

The study investigates cross-lingual relation extraction for Romanian by translating the SemEval-2010 Task 8 benchmark and evaluating Gemma 4 31B against encoder baselines. Results show that QLoRA fine-tuning improves macro F1-Score by more than 22 percentage points, reducing the cross-lingual gap from 3.3 to 1.4pp.