arxiv arXiv cs.LG · 1h ago · research

VITA clinical RAG system matches or outperforms frontier LLMs on HealthBench

from English

A purpose-built retrieval-augmented generation (RAG) system named VITA, designed for low- and middle-income settings, was evaluated against general-purpose large language models on the HealthBench benchmark. On 4,023 questions scored by a GPT-4.1 judge, VITA ranked first with 51.9% of possible rubric points, surpassing GPT-5.4, o4-mini, Gemini 3.1 Pro, and Claude Sonnet 4.6.

When re-evaluated against newer models using a neutral open-weight judge, VITA achieved parity with GPT-5.5 on mean per-question score while leading on points-weighted score and winning the most questions. The study indicates that corpus specificity improves grounding in clinical contexts, allowing specialized systems to remain competitive with frontier LLMs despite potentially lower communication polish.

Importance 2/3 Beats a top-lab benchmark arXiv cs.LG Zhipu AI Benchmark results Retrieval & RAG

Benchmarks

Benchmark	Model	Score
HealthBench	VITA	51.9%
HealthBench	GPT-5.4	46.1%
HealthBench	o4-mini	44.3%
HealthBench	Gemini 3.1 Pro	42.6%
HealthBench	Claude Sonnet 4.6	37.3%

Read original

arxiv arXiv cs.CL · just now Live

Corpus-specific VITA RAG matches or beats frontier LLMs on HealthBench

VITA, a retrieval-augmented generation system designed for low- and middle-income settings, ranks first on the English-language subset of HealthBench, outperforming GPT-5.4, o4-mini, Gemini 3.1 Pro, and Claude Sonnet 4.6.

media r/LocalLLaMA · 13d ago · 2 views

DeepSeek V4-Flash achieves 50 on ArtificialAnalysis Index

The new DeepSeek V4-Flash model has achieved a score of 50 on the ArtificialAnalysis Index. This result places it just one point below GLM-5.2 and GPT-5.6 Luna.

media MarkTechPost · 25d ago · 12 views

Kimi K3, DeepSeek V4 Pro, and GLM-5.2 compared on benchmarks, license, and serving cost

A comparison of three open-weight sparse Mixture-of-Experts models—Moonshot AI's Kimi K3, DeepSeek V4 Pro, and Zhipu AI's GLM-5.2—evaluates their capabilities, licensing terms, and serving costs for long-horizon coding and agent workloads.

arxiv arXiv cs.CL · 28d ago

BioASQ 2026 system uses cost-pragmatic re-retrieval and multi-model selection-fusion

The authors describe their BioASQ Task 14B 2026 system, which focuses on optimizing retrieval re-fetching strategies and combining answers from multiple language models. The approach employs a hybrid first-stage retrieval pipeline using BGE and BM25, coupled with an agent-driven decomposition over PubMed, Europe PMC, and iCite.

media r/LocalLLaMA · 29d ago

User builds memory pipeline on Qwen3 235B A22B scoring #1 on LongMemEval-S

A user has developed and open-sourced a long-term memory pipeline for AI assistants that runs its storage and retrieval components on the Qwen3 235B A22B Instruct 2507 model. The system achieved a score of 470 out of 500 on the LongMemEval-S benchmark, ranking first among known systems while reportedly being approximately ten times more token efficient than competitors.