The authors introduce WILDTRACE, a new benchmark designed to evaluate how well models integrate evidence dispersed naturally across distant passages in long documents. Unlike existing benchmarks that use planted facts or reverse-engineered chains, WILDTRACE relies on source-internal causal, temporal, and narrative logic.
- The benchmark comprises 481 tasks over 214 naturally occurring long-form sources, including technical incident reports and literary narratives.
- It defines seven source-internal evidence geometries to characterize the relational demands of analytical reading.
- A source-first construction pipeline mines candidate trails from document structure before questions are written.
- Each item undergoes multi-stage validation covering clue necessity, answer groundedness, rubric fidelity, contamination resistance, and answerability.
This work addresses the gap between accessing information and reasoning over naturally dispersed evidence, which is identified as a defining challenge for long-context research.