LegalHalluLens introduces a framework to audit AI hallucinations in legal contexts by analyzing typed hallucination profiles across four claim categories. It reveals a 38-40 point gap between obligation/numeric and temporal claims, and shows two systems with identical 52% hallucination rates can have opposite risk directions. The framework uses a Risk Direction Index and calibrated debate pipelines to reduce fabricated detections by 45%, offering actionable diagnostics for trustworthy legal AI deployment.
arxiv
arXiv cs.LG
·
76d ago
·
src: 77d ago
·
research
LegalHalluLens: Auditing Hallucinations in Legal AI
from English
Importance 3/3
New feature vs. leaders
New harness with differentiators
arXiv cs.LG
Mistral AI
OpenAI
xAI
AI agents
Reasoning models
Safety & alignment
Related articles
arxiv
arXiv cs.AI
·
74d ago
·
4 views
LedgerAgent: Structured State for Policy-Adherent Tool-Calling Agents
LedgerAgent introduces a structured ledger to maintain task states separately in tool-calling agents. It renders states into prompts and enforces policy constraints before tool execution, reducing policy violations and improving performance across customer-service domains.
arxiv
arXiv cs.CL
·
74d ago
·
4 views
LedgerAgent: Structured State for Policy-Adherent Tool-Calling Agents
LedgerAgent introduces a structured ledger to maintain task states separately in tool-calling agents. It renders these states into prompts and enforces policy constraints before tool execution, reducing policy violations and improving performance across customer-service domains.
arxiv
arXiv cs.CL
·
76d ago
·
3 views
LegalHalluLens: Auditing Hallucinations in Legal AI
LegalHalluLens introduces a framework to audit AI hallucinations in legal contexts by analyzing typed hallucination profiles across four claim categories. It reveals a 38-40 point gap between obligation/numeric and temporal claims, and shows two systems with identical 52% hallucination rates can have opposite risk directions. The framework uses a Risk Direction Index and calibrated debate pipelines to reduce fabricated detections by 45% and improve accountability in legal AI deployment.
arxiv
arXiv cs.AI
·
74d ago
·
3 views
Efficient and Sound Probabilistic Verification for AI Agents
A new framework enables secure, probabilistic policy enforcement for AI agents in ambiguous environments. It uses distributionally robust optimization to compute rigorous upper bounds on policy violation probabilities without assuming predicate independence. The method outperforms prior approaches on terminal and tool calling agent benchmarks, improving the security-utility trade-off.
arxiv
arXiv cs.AI
·
75d ago
·
4 views
Dual-Agent Framework for Cross-Model Verified Translation
A dual-agent framework converts natural-language experiment protocols into executable commands for robotic lab platforms. It uses a Parser Agent and a rule-based mapping engine to translate protocols, with a heterogeneous LLM Validation Agent ensuring accuracy and triggering self-correction. The framework successfully enables end-to-end autonomous execution of microplate-based experiments like the Bradford assay.