MedBench v5 introduces a dynamic, process-oriented benchmark for clinical multimodal models, featuring clinical cognitive responsiveness and atomic skills across 63 tasks. It includes stressors for degradation analysis and monitors hallucination propagation through five reasoning nodes, revealing that strong task performance does not ensure process stability.
arxiv
arXiv cs.CL
·
46d ago
·
research
MedBench v5: Dynamic Benchmark for Clinical AI
from English
Importance 2/3
New harness with differentiators
arXiv cs.CL
AI agents
Evaluation & benchmarks
Reasoning models
Related articles
arxiv
arXiv cs.LG
·
46d ago
·
1 view
LDT-FRL Framework for Cyber-Resilient IoMT
The LDT-FRL framework introduces a privacy-preserving defense system for IoMT devices, combining temporal attention, lightweight digital twins, and federated reinforcement learning. It achieves 99.66% and 99.95% accuracy on CICDDoS 2019 and TON-IoT benchmarks, with perfect F1 on the MITM class, converging 81% faster than prior methods and offering interpretable defense decisions via SHAP and Grad-CAM.
arxiv
arXiv cs.AI
·
46d ago
·
1 view
Zero-shot Procedural Mistake Detection with VLMs
A unified zero-shot framework, ZeProM, uses a pre-trained Video-Language Model to jointly perform procedural mistake detection and temporal action segmentation. It achieves up to 4.4 point improvement in EDA and 2.0 point in F1@.5 on EgoPER tasks, matching or exceeding supervised methods without task-specific training.
arxiv
arXiv cs.CL
·
47d ago
·
1 view
Self-Evolution of Tool-Calling Agents via Divergence-Point Preference Learning
ToolGraph enhances multi-turn tool-using agents by integrating schema topology, transition weights, and history-aware controls. Training with DPO on 161 divergence-point preference pairs improves performance: ToolGraph+DPO achieves a 16.8% relative reward gain over baseline, especially in airline and retail tasks, with reward positivity emerging as the key diagnostic signal.
arxiv
arXiv cs.AI
·
51d ago
Optimal Order in Multi-Agent Systems Framework
A new framework analyzes multi-agent systems by modeling agent influence and response functions. It derives macroscopic properties like power, entropy, and order, and identifies an optimal level of synchronization that balances productivity, stability, and adaptability. The study shows that order and system properties are task-dependent and context-relative.
arxiv
arXiv cs.CL
·
51d ago
·
1 view
H-RePlan: Hierarchical Recovery for Cross-Device Agent Systems
H-RePlan introduces a hierarchical replanning framework that separates device-local strategy recovery from global orchestrator replanning. It outperforms existing baselines by achieving higher completion and instruction adherence, with reduced token cost, through scope-aware recovery in multi-device agent systems.