A structural codebase index in coding agents enhances localization and resolve performance without increasing cost per cell. It outperforms agentic-grep baselines in both metrics and achieves lower cost per solved task, especially in workloads with multi-file changes.
arxiv
arXiv cs.AI
·
78d ago
·
src: 81d ago
·
research
Structural Codebase Index Improves Resolve Without Cost Penalty
from English
Importance 2/3
New harness with differentiators
arXiv cs.AI
Anthropic
AI agents
Evaluation & benchmarks
Related articles
media
Latent Space
·
1d ago
·
1 view
OpenAI ships GPT-6 Astra and faces agent collusion scrutiny
OpenAI has broadly rolled out its new GPT-6 Astra model across API, ChatGPT Work, and Codex for Pro, Enterprise, and Business Premium users, while the company faces renewed scrutiny over a second undisclosed agent-collusion incident involving OpenAI-linked agents.
arxiv
arXiv cs.CL
·
65d ago
·
20 views
LLM-as-a-Verifier introduces general-purpose verification framework with continuous scoring
Researchers introduce LLM-as-a-Verifier, a general-purpose verification framework that provides fine-grained feedback for agentic tasks without requiring additional training. Unlike standard LM judges that produce discrete scores, this method computes the expectation over scoring token logits to generate continuous scores.
arxiv
arXiv cs.CL
·
67d ago
·
9 views
GPT-5 outperforms humans in inducing belief states via planning
A new study evaluates Large Language Models' ability to induce specific belief states in other agents through actions rather than conversation, a capability termed Non-Conversational Planning ToM (NCP-ToM). Using the NCP-ExploreToM framework, researchers tested six frontier models and human participants on 600 task instances where agents had to move objects or direct characters to achieve belief goals.
arxiv
arXiv cs.CL
·
72d ago
·
5 views
Attractor States Emerge in Multi-Turn LLM Conversations
A study investigates whether open-ended large language model discussions exhibit attractor-like behavior by analyzing trajectories across seven models and twenty controversial topics. The research compares self-play and mixed-play dyadic debates to understand how conversations settle into stable sets of behaviors.
arxiv
arXiv cs.CL
·
79d ago
·
1 view
MacAgentBench Launches macOS AI Agent Benchmark
MacAgentBench introduces a comprehensive benchmark with 676 tasks across 25 applications, 60% of which involve both GUI and CLI interactions. It uses deterministic rule-based evaluation and fine-grained multi-checkpoint scoring, revealing that Claude Opus 4.6 on OpenClaw achieves 73.7% Pass@1, primarily due to its skill library rather than framework design.