FCPA training objective improves LLM generator-validator alignment

Researchers introduce Frequency-Corrected Prompt Alignment (FCPA), a new formulation of generator-validator consistency that addresses the G-V gap in large language models. The method applies a principled correction for utterance frequency, resolving issues where generators assign low likelihood to valid strings simply because they are a priori unlikely.

• FCPA implements a training objective based on frequency-corrected G-V consistency for real-world LLMs.
• Training with FCPA substantially improves both G-V consistency and generator performance compared to prior methods.
• The approach yields gains of up to +27pp in Pearson correlation on IFEval and HumanEval benchmarks.
• Validator quality is preserved across all evaluated tasks despite the improvements in generator performance.

The authors consider this important because it provides a robust mechanism for aligning generation and validation processes, leading to more reliable model outputs without degrading the validator's capabilities.