Researchers address the issue of timestamp drift in autoregressive ASR systems, where generated timestamps diverge from audio during long non-speech spans. They propose REDDIT, a lightweight two-stage post-training framework that corrects these timestamps while preventing catastrophic forgetting of non-target behaviors.
- The method uses a replay-based distribution editing approach that edits timestamp targets under the model's own decoder context while matching the frozen base distribution on non-timestamp tokens.
- Correction supervision is constructed without human transcripts or annotations by combining VAD-trimmed speech spans with inserted non-speech gaps and known concatenation offsets.
- On Whisper-tiny, using only 34.9 hours of targeted correction audio and updating just 1.6% of model parameters, the framework raises long-gap mIoU from 38.7% to 95.0%.
- It reduces mixed-gap out-of-domain AAS from 2752 ms to 223 ms while preserving CV-en MER at 41.3%, significantly outperforming ordinary SFT decoder tuning which resulted in a 524.2% MER.
This approach enables accurate timestamped transcription without frame-level aligners or inference-time post-processing, effectively correcting drift while maintaining general ASR performance.