The paper introduces future confidence distillation, a method that trains predictors on pre-solution hidden representations using teacher confidence estimates derived from post-solution correctness probes. This approach allows the model to anticipate confidence-related information before answer generation is complete.
- Post-solution confidence is consistently better calibrated and more discriminative than pre-solution Feeling-of-Knowing estimates.
- Linear probes trained on hidden representations recover substantially richer confidence-related information than models explicitly verbalize.
- Distilled predictors require only pre-solution representations for inference, remain highly sample efficient, and transfer across datasets within the same domain.
The authors consider this important because it enables significantly more reliable yet low-cost confidence estimation for downstream decisions in confidence-aware systems.