Act2Answer introduces a lightweight protocol to assess commonsense and world knowledge retention in VLA models by requiring agents to answer questions through object placement actions. A large-scale study of 7 VLA models and 9 VLM baselines reveals that VLAs perform well on simple concepts but show larger gaps on rich semantic categories compared to their source VLMs, with VQA co-training improving knowledge retention and peak answer-relevant signals observed in middle VLA layers.
arxiv
arXiv cs.LG
·
52d ago
·
research
Act2Answer Evaluates Knowledge Retention in Vision-Language-Action Models
from English
Importance 2/3
New harness with differentiators
arXiv cs.LG
Allen AI
Hugging Face
Microsoft Research
AI agents
Evaluation & benchmarks
Reasoning models
Related articles
arxiv
arXiv cs.CL
·
51d ago
·
1 view
H-RePlan: Hierarchical Recovery for Cross-Device Agent Systems
H-RePlan introduces a hierarchical replanning framework that separates device-local strategy recovery from global orchestrator replanning. It outperforms existing baselines by achieving higher completion and instruction adherence, with reduced token cost, through scope-aware recovery in multi-device agent systems.
arxiv
arXiv cs.AI
·
53d ago
EAGG: Embodiment-Aligned Grasp Generation via Geometry-Aware Graph Conditioning
EAGG introduces a grasp generator that aligns embodiment structure within a shared model using topology-aware graphs and geometry-aware tokens. It achieves 56.17% average grasp success on MultiGripperGrasp, matching specialized models within 1.10 percentage points and reducing median contact distance from 0.239 cm to 0.189 cm.
media
r/LocalLLaMA
·
47d ago
·
1 view
Best local models for reasoning in agentic AI
The creator of EverFern asks which local models work best for agentic workflows and browser/computer use. They note that model intelligence is rarely the bottleneck, with reliability and recovery systems being more critical than model choice.
arxiv
arXiv cs.CL
·
47d ago
·
1 view
Self-Evolution of Tool-Calling Agents via Divergence-Point Preference Learning
ToolGraph enhances multi-turn tool-using agents by integrating schema topology, transition weights, and history-aware controls. Training with DPO on 161 divergence-point preference pairs improves performance: ToolGraph+DPO achieves a 16.8% relative reward gain over baseline, especially in airline and retail tasks, with reward positivity emerging as the key diagnostic signal.
arxiv
arXiv cs.CL
·
47d ago
·
1 view
Beaver: Agent Harness for Scientific Curation from Multimodal Sources
Beaver is an agent harness that extracts structured information from scientific papers by integrating multimodal evidence tooling, task scaffolding, and artifact-grounded autoresearch. It achieves 81.0 on the Gold-Referenced Attribute Score, outperforming frontier agents by over 23 points, with key gains on high-value attributes requiring cross-modal reasoning.