IBM releases Granite Speech 5.0 Turbo CTC models with over 12,600 RTFx speed

IBM has released two new compact, 470M-parameter English speech recognition models in the Granite Speech family: `granite-speech-5.0-470m-turboctc` and `granite-speech-5.0-470m-turboctc-nc`. These encoder-only models achieve unprecedented transcription speeds of over 12,600 RTFx on an NVIDIA H200 GPU while maintaining high accuracy.

• The non-commercial model (`-nc`) is trained on additional data and carries a CC-BY-NC-SA-4.0 license, scoring 4.85% WER on OpenASR public test sets.
• The Apache 2.0 licensed model scores 5.00% WER and ranks fifth in accuracy on the FFASR Leaderboard while being the fastest two models.
• Both models use a stack of 16 Conformer blocks with chunkwise attention and generate 12.5 tokens per second, offering over 20x faster throughput than previous Granite Speech models.
• The architecture relies on three stages of 2x subsampling to reduce the log Mel spectrogram rate from 100 frames per second to 12.5 tokens per second.

The new encoder-only design provides a small memory footprint ideal for speech-to-text tasks on edge devices, though it sacrifices capabilities like speech translation found in prior LM-equipped models.