Researchers propose Unified Gradient Projection (UGP), a method to mitigate catastrophic forgetting when fine-tuning large pretrained ASR models like Whisper on low-resource languages. UGP constrains parameter updates using reference gradients from language-balanced replay within a unified projection space, effectively equalizing per-language contributions and reducing dominant-language bias.
- The approach combines gradient-level projection with data-level replay to yield complementary gains in stability and plasticity.
- It addresses cross-task interference where dominant languages typically bias optimization in multilingual settings.
- On Whisper-large-v3, UGP achieves near-zero average forgetting across diverse low-resource language groups and model scales.
This method enables effective adaptation for low-resource languages while substantially mitigating the loss of previously learned capabilities.