A study introduces lightweight, interpretable metrics that sharpen the boundary between clinically significant errors and harmless variations in radiology reports. These metrics outperform large medical LLMs and rival proprietary models, with one-pass training proven effective for cost-sensitive deployment. The two-pass setting fails to consistently improve performance and shifts focus from error detection to robustness.
arxiv
arXiv cs.CL
·
52d ago
·
research
LLM-based Metrics Improve Clinical Significance Evaluation in Radiology
from English
Related articles
arxiv
arXiv cs.AI
·
51d ago
·
3 views
CRAX: Fast Safe Reinforcement Learning Benchmarking
CRAX introduces a high-fidelity, accelerated safety benchmark for reinforcement learning using MuJoCo XLA. It achieves up to 100x speedups over CPU-based benchmarks via vectorization and hardware acceleration, featuring six environment suites and three agent-specific tasks across three difficulty levels. Evaluation of six safe RL methods shows no single approach dominates, highlighting trade-offs between performance and safety, with curriculum learning and safety transfer improving results.
arxiv
arXiv cs.LG
·
51d ago
·
3 views
CRAX: Fast Safe Reinforcement Learning Benchmarking
CRAX introduces a high-fidelity, fast safety benchmark for reinforcement learning using MuJoCo XLA. It achieves up to 100x speedups over CPU-based benchmarks via vectorization and hardware acceleration, featuring six environment suites and three agent-specific tasks across three difficulty levels. Evaluation of six safe RL methods shows no single approach dominates, highlighting trade-offs between performance and safety, with curriculum learning and safety transfer improving results.
arxiv
arXiv cs.CL
·
45d ago
·
1 view
Real-Time Voice AI Hears but Does Not Listen
A study evaluates four leading production real-time voice systems: OpenAI's GPT Realtime 2, Google's Gemini 3.1 Flash Live, and Alibaba's Qwen3.5 Omni Plus and Omni Flash. The research focuses on tasks where both words and vocal delivery convey meaningful information across three consequential scenarios. All four systems act on the literal words rather than the voice, leading to errors such as ending calls with crying users who insist nothing is wrong or approving wire transfers made in frightened voices. Surprisingly, this disconnect is often not a failure of perception, as three of the four systems can reliably identify distress, fear, or sarcasm when asked directly. Despite this awareness, the models ignore these emotional cues during decision-making, exhibiting what the authors term the 'emotional intelligence gap.' The study also notes that systems estimate accent and age based on word biases rather than acoustic properties. Prompting the systems to explicitly attend to vocal delivery improves performance only partially and inconsistently. These findings suggest current real-time voice AI behaves as if speech were reduced to a transcript, warranting caution in settings where tone is critical.
arxiv
arXiv cs.AI
·
51d ago
MACR: Explicit Conflict Resolution for LLM Inference
MACR introduces a multi-agent reasoning framework to resolve knowledge conflicts in LLM inference by jointly assessing internal and external knowledge. It uses semantic entropy to measure confidence and employs three specialized agents to induce rules, detect conflicts, and resolve inconsistencies across contexts. Empirical results show MACR outperforms state-of-the-art methods and provides interpretable conflict resolutions.
arxiv
arXiv cs.LG
·
51d ago
De-biased VLM-as-3D-Judge Protocol for Furniture Generation
A de-biased VLM-based judge protocol specializes TRELLIS on furniture generation using lightweight adaptation. The protocol addresses failure modes like image overload and geometry-hiding, with calibration showing 0.83–1.0 win rates and base-vs-base symmetry at 0.5. Among six adaptation methods, conditioner repair under severe degradation achieves parity with the base model, while no method exceeds a 65% win-rate target.