Study finds reasoning in English improves uncertainty estimation in low-resource languages

A large-scale evaluation of uncertainty estimation (UE) methods across 22 languages reveals that prompting models to reason in English significantly enhances performance, particularly for low-resource languages. The study compares nine open and closed-box UE methods using human-curated Q&A datasets while eliciting long-form reasoning.

• Prompting models to reason in English substantially improves UE performance even when questions are in low-resource languages, indicating the reliability bottleneck lies in generation rather than understanding.
• Using English for reasoning closes the UE performance gap between low and high-resource languages, demonstrating that the language of generation matters more than the question language.
• The optimal UE method depends on model scale: open-box probability-based methods outperform alternatives at smaller scales, while closed-box self-verbalized uncertainty is superior at larger scales.

The authors provide guidance on threshold selection for selective prediction to help calibrate abstention in multilingual settings.