CRAX introduces a high-fidelity, accelerated safety benchmark for reinforcement learning using MuJoCo XLA. It achieves up to 100x speedups over CPU-based benchmarks via vectorization and hardware acceleration, featuring six environment suites and three agent-specific tasks across three difficulty levels. Evaluation of six safe RL methods shows no single approach dominates, highlighting trade-offs between performance and safety, with curriculum learning and safety transfer improving results.
arxiv
arXiv cs.AI
·
51d ago
·
research
CRAX: Fast Safe Reinforcement Learning Benchmarking
from English
Importance 3/3
New feature vs. leaders
New harness with differentiators
arXiv cs.AI
Google DeepMind
Microsoft Research
NVIDIA
Evaluation & benchmarks
Reasoning models
Safety & alignment
Related articles
arxiv
arXiv cs.LG
·
51d ago
·
3 views
CRAX: Fast Safe Reinforcement Learning Benchmarking
CRAX introduces a high-fidelity, fast safety benchmark for reinforcement learning using MuJoCo XLA. It achieves up to 100x speedups over CPU-based benchmarks via vectorization and hardware acceleration, featuring six environment suites and three agent-specific tasks across three difficulty levels. Evaluation of six safe RL methods shows no single approach dominates, highlighting trade-offs between performance and safety, with curriculum learning and safety transfer improving results.
arxiv
arXiv cs.CL
·
52d ago
LLM-based Metrics Improve Clinical Significance Evaluation in Radiology
A study introduces lightweight, interpretable metrics that sharpen the boundary between clinically significant errors and harmless variations in radiology reports. These metrics outperform large medical LLMs and rival proprietary models, with one-pass training proven effective for cost-sensitive deployment. The two-pass setting fails to consistently improve performance and shifts focus from error detection to robustness.
arxiv
arXiv cs.AI
·
51d ago
UFP4: Uniform 4-Bit Training Overcomes Shrinkage Bias in LLM Pretraining
A study identifies shrinkage bias in E2M1-based FP4 formats due to geometric asymmetry, causing multiplicative error accumulation and training instability. The proposed UFP4 recipe uses uniform E1M2/INT4 grids and applies Random Hadamard Transform to all GEMMs, achieving lower loss degradation than E2M1 baselines in large-scale LLM pretraining. The authors recommend E1M2/INT4 as a first-class training primitive for future accelerators.
arxiv
arXiv cs.AI
·
51d ago
MACR: Explicit Conflict Resolution for LLM Inference
MACR introduces a multi-agent reasoning framework to resolve knowledge conflicts in LLM inference by jointly assessing internal and external knowledge. It uses semantic entropy to measure confidence and employs three specialized agents to induce rules, detect conflicts, and resolve inconsistencies across contexts. Empirical results show MACR outperforms state-of-the-art methods and provides interpretable conflict resolutions.
arxiv
arXiv cs.AI
·
51d ago
·
1 view
Finetuning VLA Models Requires Fewer Layers Than Thought
Vision-Language-Action models show severe layer-wise redundancy despite large parameter counts. A training-free compression method using Centered Kernel Alignment removes twin layers, reducing model depth by up to 50% and enabling 40-50% faster training and up to 30% faster inference without performance loss, validated across simulation and real-world robotic tasks.