CARD removes audio encoder for efficient captioning by distilling CLAP-HTSAT

The authors introduce CARD, an encoder-free audio captioning model that eliminates the inference cost of frozen audio encoders by using a 13.2M projector to feed a frozen LLM with merged LoRA adapters.

• CARD distills a pretrained CLAP-HTSAT teacher into the system rather than relying on the encoder during inference.
• The method routes perceptual stages to the projector and semantic stages to the LLM.
• This approach improves CIDEr-D by +12.18 over an LLM-only distilled model on AudioCaps.
• It achieves a score of 55.4 on Clotho, compared to a 66.4 upper bound with an encoder kept.

The work demonstrates that strategically placing a teacher's knowledge across components matters as much as its presence for performance.