EfficientRollout introduces a self-speculative decoding framework that reduces rollout and end-to-end latency by up to 19.6% and 12.7% respectively, without compromising final model quality. It uses a quantized drafter derived from the target model and integrates a system-aware toggle policy to avoid compute-bound regimes, enabling effective speculation during evolving policy generations.
arxiv
arXiv cs.LG
·
55d ago
·
research
EfficientRollout: System-Aware Self-Speculative Decoding for RL Rollouts
from English
Importance 3/3
New feature vs. leaders
New harness with differentiators
arXiv cs.LG
Microsoft Research
Meta AI
OpenAI
AI agents
Inference efficiency
Reasoning models
Related articles
arxiv
arXiv cs.CL
·
57d ago
·
1 view
KVEraser: Efficient Localized Context Erasing in LLMs
KVEraser enables efficient localized context erasing in large language models by replacing only the KV cache states of an erased span with learned steering states. It achieves near-full-recomputation performance on in-domain tasks across 1K to 32K context lengths, with only a 24% latency increase, and outperforms other approximate methods in long-document QA with 3--4x speedup over full recomputation.
media
AI News (smol.ai)
·
17h ago
Meta releases Muse Glimmer 30B open-weight model; Anthropic improves Riemann bound
Meta released Muse Glimmer, a 30B dense multimodal model optimized for local agent deployment under Apache 2.0, while also promising future weights for Muse Spark 1.2.
media
AI News (smol.ai)
·
5d ago
Meta's Muse Spark 1.2 and OpenAI's GPT-5.6 Sol updates
Meta announced that its Muse Spark 1.2 model achieved gold-medal performance in five STEM Olympiads and entered the top 5 on the Vals Index at $0.69/test, reportedly 3x cheaper than Kimi. Meta attributed these gains to multi-agent orchestration with parallel reasoning, noting perfect theory scores at APhO and IPhO without external tools.
arxiv
arXiv cs.AI
·
54d ago
·
3 views
AutoPass: Evidence-Guided LLM Agents for Compiler Performance Tuning
AutoPass uses runtime and compiler evidence to guide LLM-generated optimization decisions, outperforming expert heuristics and classical autotuning methods. It achieves geometric-mean speedups of 1.043x on x86-64 and 1.117x on ARM64 systems without prior training or fine-tuning.
arxiv
arXiv cs.LG
·
54d ago
·
1 view
LLM-based Hierarchical Control in Multi-Agent Games
A hierarchical system using a pretrained LLM to select RL skill policies outperforms flat RL in a 2v2 King of the Hill environment. It matches hand-crafted behavior tree performance in win rate and is perceived as more human-like by 60% of users, highlighting effective coordination and adaptability without manual rule design.