WorldLines introduces a project-driven benchmark for long-horizon embodied household assistance, capturing extended household traces with dialogues, actions, and state changes. It enables evidence-linked samples for Memory QA and Embodied Task Planning, and proposes ObsMem, an observer-grounded memory framework that supports visibility-aware memories and state-aware decisions. Experiments highlight challenges in partial observability and memory translation, with ObsMem providing a stronger reference architecture for such settings.
arxiv
arXiv cs.AI
·
55d ago
·
research
WorldLines: Benchmarking Long-Horizon Embodied Agent Memory
from English
Importance 3/3
New feature vs. leaders
New harness with differentiators
arXiv cs.AI
Microsoft Research
Google DeepMind
OpenAI
AI agents
Evaluation & benchmarks
Reasoning models
Related articles
arxiv
arXiv cs.CL
·
54d ago
·
2 views
AtomMem: Simple and Effective Memory System for LLM Agents
AtomMem introduces a memory system that stores high-value atomic facts from long-form interactions. It uses hierarchical event structures and temporal profiles to capture coherent episodic contexts and track evolving user attributes, enabling stable and efficient memory evolution. Experiments on the LoCoMo benchmark show AtomMem achieves state-of-the-art performance in reasoning tasks.
arxiv
arXiv cs.CL
·
54d ago
GEMS: Geometric Constraints Enable Multi-Semantic Superposition in LLMs
GEMS enables training-free superposition of multiple semantic directions in LLMs by addressing distributional deviation and directional interference through geometric constraints. On GSM8K, it maintains 98% accuracy with three non-mathematical directions, while unconstrained addition drops to 4%; on Wikitext-2, it increases PPL by only 2.2%.
arxiv
arXiv cs.AI
·
55d ago
·
5 views
ScenA: Reference-Driven Multi-Speaker Audio Scene Generation
ScenA conditions a text-to-audio foundation model on multiple reference voices and a natural language scene prompt to generate realistic multi-speaker conversations. It addresses the 'Reference Shortcut' issue by using a high-noise-biased training schedule, ensuring speaker assignment relies on text prompts rather than acoustic similarity. Evaluated on CoVoMix2-Dialogue, Scen- A outperforms existing systems in speaker-binding and produces rich, naturalistic audio with overlapping speech and ambient noise.
arxiv
arXiv cs.CL
·
55d ago
·
2 views
Turing-RL: Learning User Simulators with Turing Rewards
Turing-RL introduces a reinforcement learning method using an LLM judge to evaluate how indistinguishable generated responses are from real user inputs. It outperforms baseline methods in both LLM and human evaluations across chat and Reddit forum domains, demonstrating that optimizing for indistinguishability improves user simulator performance.
arxiv
arXiv cs.CL
·
55d ago
EARS Framework Enhances Multi-Agent System Reliability
EARS introduces explanatory abstention in sub-agents to improve reliability in large-scale multi-agent systems. By providing actionable failure rationales to coordinators, EARS increases the overall response pass rate from 68.5% to 78.9% in a production e-commerce assistant.