The authors introduce TRIAGE, a framework designed to evaluate the trustworthiness of Graph-based Retrieval-Augmented Generation (Graph-RAG) systems that rely on automatically built knowledge graphs. Unlike traditional evaluation methods that only check final outputs, TRIAGE instruments three distinct stages: KG Implementation, KG Validation by expert, and KG Usage.
- The framework attaches stage-specific metrics to each phase, including triple confidence and source coverage for implementation, graph-level structural quality for validation, and retrieval coverage and faithfulness for usage.
- Deployed metrics require no gold annotations, while those needing references serve only as offline calibration.
- These metrics form a diagnostic chain where the first broken link localizes the failure to a specific stage: extraction, graph schema, or retrieval.
This approach allows users to identify exactly where failures occur in the pipeline and apply targeted remedies rather than just observing incorrect final answers.