Topic · Voice & audio
media Hugging Face Forums · 27d ago

Developer explores orchestration patterns for production voice AI agents

A developer on the Hugging Face forums seeks to understand how production platforms like Bland.ai, Retell, and Vapi manage prompt orchestration without relying on massive, hand-written system prompts. The author describes their current implementation using FastAPI, Sarvam STT/TTS, and Pipecat SmartTurn V3, noting that manual state-wise injection leads to growing complexity and edge-case failures.

arxiv arXiv cs.CL · 24d ago

Auditing Protocol-Level Shortcuts in Large Audio Language Model Judges for Speech Evaluation

A study audits protocol-level shortcuts in large audio-language models (LALMs) used as automatic judges for speech evaluation, revealing that high agreement with human ratings does not guarantee verdicts are grounded in the actual audio. The research examines three deployment protocols: feature-blueprint judging, reference-conditioned judging, and pairwise A/B comparison across six judges and four attributes.

arxiv arXiv cs.CL · 25d ago

Unified Gradient Projection reduces catastrophic forgetting in multilingual ASR

Researchers propose Unified Gradient Projection (UGP), a method to mitigate catastrophic forgetting when fine-tuning large pretrained ASR models like Whisper on low-resource languages. UGP constrains parameter updates using reference gradients from language-balanced replay within a unified projection space, effectively equalizing per-language contributions and reducing dominant-language bias.