WILDTRACE introduces benchmark for natural evidence trails in long-context reasoning

The authors introduce WILDTRACE, a new benchmark designed to evaluate how well models integrate evidence dispersed across distant passages in long documents. Unlike existing benchmarks that rely on planted facts or reverse-engineered chains, WILDTRACE uses 481 tasks derived from 214 naturally occurring sources, such as technical incident reports and literary narratives.

• The benchmark defines seven source-internal evidence geometries based on causal, temporal, and narrative logic.
• A source-first pipeline mines candidate trails from document structure before questions are written.
• Items undergo multi-stage validation for clue necessity, answer groundedness, rubric fidelity, contamination resistance, and answerability.

This work addresses the gap between accessing information and reasoning over naturally dispersed evidence, which is critical for real-world high-stakes analytical tasks.