ProvenanceGuard introduces a source-aware verifier for MCP-based LLM agents that detects cross-source conflation by routing claims to specific evidence sources and comparing stated attribution with actual source ownership. It achieves block F1 of 0.802 and source accuracy of 0.858 on 260 source-eligible claims, outperforming source-blind baselines, and detects all injected attribution swaps in 50 clinical probes.
arxiv
arXiv cs.CL
·
57d ago
·
research
ProvenanceGuard: Source-Aware Factuality Verification for MCP-Based LLM Agents
from English
Importance 3/3
Beats a top-lab benchmark
New feature vs. leaders
arXiv cs.CL
OpenAI
Anthropic
Google DeepMind
AI agents
Reasoning models
Retrieval & RAG
Related articles
arxiv
arXiv cs.AI
·
57d ago
ProvenanceGuard: Source-Aware Factuality Verification for MCP-Based LLM Agents
ProvenanceGuard introduces a source-aware verifier for MCP-based LLM agents that detects cross-source conflation by routing claims to specific evidence sources and comparing stated attribution with actual source ownership. It achieves block F1 of 0.802 and source accuracy of 0.858 on 260 source-eligible claims, outperforming source-blind baselines, and detects all injected attribution swaps in 50 clinical probes.
arxiv
arXiv cs.CL
·
56d ago
·
2 views
Turing-RL: Learning User Simulators with Turing Rewards
Turing-RL introduces a reinforcement learning method using an LLM judge to evaluate how indistinguishable generated responses are from real user inputs. It outperforms baseline methods in both LLM and human evaluations across chat and Reddit forum domains, demonstrating that optimizing for indistinguishability improves user simulator performance.
arxiv
arXiv cs.CL
·
56d ago
·
1 view
Decoupling Search from Reasoning in LLM Agents
Decoupled Search Grounding (DSG) separates search functionality from reasoning models, enabling vendor-agnostic, tunable, and reusable search grounding. DSG achieves near-native accuracy on SimpleQA with 91% lower search cost and 99.4% warm-cache hit rate, while reducing latency by 68% and preserving concise output contracts.
arxiv
arXiv cs.AI
·
56d ago
Decoupling Search from Reasoning in LLM Agents
Decoupled Search Grounding (DSG) separates search functionality from reasoning models, enabling vendor-agnostic, tunable, and reusable search grounding. DSG achieves near-native accuracy on SimpleQA with 91% lower search cost and 99.4% warm-cache hit rate, while reducing latency by 68% and preserving concise output contracts.
arxiv
arXiv cs.AI
·
56d ago
RTSGameBench: An RTS Benchmark for Strategic Reasoning
RTSGameBench addresses limitations in existing RTS benchmarks by offering diverse gameplay, targeted competency diagnosis, and self-evolving scenario generation. It evaluates vision-language models in strategic reasoning under uncertainty, revealing that state-of-the-art models struggle with multiagent coordination and large-scale tasks.