ThinkProbe is a framework for the structural analysis of large language model reasoning traces, converting them into directed Thought Graphs with eight node types and six edge types. It derives a 19-metric five-dimensional cognitive profile through a fully non-generative pipeline combining rule-based segmentation and discriminative semantic linking.
- Applied to 4,200 traces from seven native reasoning models across 200 open-ended questions and ten cognitive domains.
- Reveals that reasoning structure is a stable, model-level property where between-model variance exceeds between-domain variance by up to fourfold.
- Structure dimension shows genuine sensitivity to question domain, exposing qualitatively distinct cognitive profiles invisible to accuracy-based evaluation.
The study demonstrates that structural profiling provides insights into model cognition that are not captured by traditional accuracy metrics.