CARE-PPO enables joint learning of accurate predictions and reliable confidence in language-based quantitative tasks

Researchers introduce CARE-PPO, a reinforcement learning framework that connects loss prediction for uncertainty estimation with actor-critic PPO fine-tuning. This approach allows for the joint learning of accurate numerical estimates and reliable confidence signals in language-based quantitative prediction from unstructured inputs.

• CARE-PPO utilizes a Confidence-Aligned Reward for Estimation to provide dense error-aware feedback to the actor while aligning the critic with prediction quality.
• The critic is repurposed as a confidence estimator during inference, outperforming logit-based and verbalized baselines in alignment.
• Evaluated on healthcare and finance tasks using Qwen-3 models (4B and 8B), CARE-PPO achieves strong quantitative performance under out-of-distribution settings.
• The method reduces task-specific overfitting on general instruction-following prompts, demonstrating broader generalization advantages of RL fine-tuning.

CARE-PPO addresses the critical need to know when predictions can be trusted by providing significantly better-aligned confidence estimates than existing baselines.