Revisiting the Relation Between Language Model Perplexity and ASR Word Error Rate for Modern End-to-End Speech Recognition

This paper revisits the historical assumption that language model (LM) perplexity (PPL) serves as a linear proxy for automatic speech recognition (ASR) word error rate (WER) in log-log space. It examines whether external LMs still improve modern end-to-end ASR systems and how internal language modeling affects this relationship.

• The study investigates if the PPL-WER relation remains linear for modern systems that already possess internal language modeling capacity.
• It analyzes how encoder context length influences the observed correlation between perplexity and error rate.
• The research explores how large language model (LLM) perplexities fit into trends previously established by standard neural LMs.
• Internal language modeling (ILM) subtraction is shown to change the observed PPL-WER relation, indicating that the decoder's internal LM must be considered when interpreting external LM quality.

The findings suggest that the decoder's internal language modeling capacity significantly impacts how external LM quality affects ASR performance.