Code-MUE measures code LLM uncertainty via execution-based semantic graphs

Researchers introduce Code-MUE, a purely black-box framework that quantifies the uncertainty of Code Large Language Models by analyzing execution-based Semantic Interaction Graphs. This approach bridges the gap between syntax and semantics by grounding uncertainty in observable runtime behavior rather than superficial textual similarity.

• Code-MUE calculates the Von Neumann entropy of the solution space to quantify global semantic diversity.
• It addresses the limitation of existing methods that fail to capture code fragility where syntactic variation does not imply semantic divergence.
• The framework is designed to work with closed-source models where white and grey-box techniques are inapplicable.

A large-scale empirical study across eight state-of-the-art LLMs demonstrates that Code-MUE achieves a strong negative correlation with functional correctness, significantly outperforming lexical and embedding-based baselines.