Eigenvalue Calibration for Semantic Embeddings of Large Language Models

Researchers propose a novel framework to calibrate the eigenvalues of semantic embeddings in large language models, addressing the gap where conventional classification calibration methods do not apply. The approach treats LLMs with semantic embeddings as density matrix predictors and applies temperature scaling to their eigenvalues.

• Interprets LLMs combined with semantic embeddings as density matrix predictors.
• Applies temperature scaling to eigenvalues for calibration.
• Establishes entropy-risk equivalence under calibration.
• Derives a central calibration inequality specific to eigenvalues.
• Proves that temperature-scaled eigenvalues optimize calibration when minimizing proper score risks.

Experiments on real-world settings show that current LLMs are systematically overconfident, validating the theoretical findings and advancing uncertainty quantification for semantic embeddings.