FlowEdit enables frozen flow-matching TTS models to adapt pronunciation corrections over time using latent edits in text embeddings. It stores corrections in a Modern Hopfield Network and retrieves them via soft attention with similarity gating, reducing phoneme error rates by 92.7% on 312 multilingual proper nouns while preserving general-speech quality. Corrections take about 15 seconds to complete on a single GPU.
A new method adapts DAAM to speech diffusion models, analyzing how style captions influence TTS waveforms. It reveals style tokens have lower temporal variance than content tokens, with style attention correlating to pitch and energy, and peak style conditioning in early layers where attention entropy is minimized, indicating maximal selectivity.
A pretrained speech classifier is repurposed as a backbone for guided diffusion-based speech generation. By attaching a lightweight subnetwork and training it under denoising score matching, the approach achieves high speech quality with reduced memory and computational cost, using a single model instead of two separately trained components.
A hybrid two-stage diffusion transformer architecture enables efficient and accurate instruction-guided audio editing. It uses coarse-to-fine semantic alignment via joint attention at low resolution, followed by refined editing with alternating joint and cross-attention at high resolution. The method achieves better performance on complex editing tasks with improved efficiency and a compact model.
PASQA is a speech quality assessment model designed to evaluate pitch-accent correctness in synthetic Japanese speech. It uses a dataset with controlled accent errors and achieves high accuracy in ranking accent-error severity, outperforming conventional models and aligning better with human judgments.
A new framework assesses pronunciation using only native speech data, without labeled errors. It uses speech token surprisal and transcript-guided alignment to detect phonotactic deviations, achieving performance close to supervised methods on multiple datasets.
MOS prediction models accurately capture acoustic degradation but fail to detect prosodic errors and speaker-specific characteristics like pitch and speaking rate. Human listeners perceive significant quality drops for these perturbations, while models show strong biases in fundamental frequency and lack sensitivity to speaking rate and F0 variability.
A new framework uses autoencoder with contrastive learning to analyze segment-level Mandarin speech for cognitive impairment detection. It achieves stable, competitive performance across four datasets, with significant improvements in three-class classification, especially under limited labeled data conditions.
PASQA is a speech quality assessment model designed to evaluate pitch-accent correctness in synthetic Japanese speech. It uses a dataset with controlled accent errors and incorporates self-supervised learning, mora-conditioned fusion, ranking loss, and accent-error localization to achieve high accuracy in detecting accent errors across speakers, outperforming conventional models in alignment with human judgments.
ReNikud introduces a novel audio-supervised approach for Hebrew grapheme-to-phoneme conversion, using weak audio supervision and a pseudo-vocalization architecture. It outperforms prior state-of-the-art methods on Hebrew G2-Ph benchmarks and the new MILIM benchmark, enabling more natural spoken Hebrew in text-to-speech applications.
A user is seeking recommendations for real-time speech-to-text tools with diarization capabilities, asking about alternatives to Wispr Flow and MacParakeet, which uses Parakeet and Whisper models. They inquire whether newer models have emerged to support real-time performance.
ScenA conditions a text-to-audio foundation model on multiple reference voices and a natural language scene prompt to generate realistic multi-speaker conversations. It addresses the 'Reference Shortcut' issue by using a high-noise-biased training schedule, ensuring speaker assignment relies on text prompts rather than acoustic similarity. Evaluated on CoVoMix2-Dialogue, Scen- A outperforms existing systems in speaker-binding and produces rich, naturalistic audio with overlapping speech and ambient noise.
A learnable residual speech-to-spike encoder is jointly trained with a Recurrent Leaky Integrate-and-Fire network, achieving up to 94.97% accuracy on the Google Speech Commands v2 benchmark. A 35k-parameter version reaches 89.8%, outperforming prior methods with far fewer parameters, and shows task-aligned spike representations that improve class separability.
A study evaluates speech-driven MFCC features and an HMM-DNN model with attention mechanisms for Chinese dialect discrimination. The approach combines word-level embeddings and MFCC features using a CNN, achieving superior performance on benchmark dialect corpora compared to existing methods.
The Inflect-Nano-v1 model is the second smallest publicly released TTS model after TinyTTS, with 4.63M total parameters. It performs surprisingly well for its size, running locally on low-end devices and offering a baseline for tiny speech synthesis in embedded or offline applications.
A developer spent 12 months building a local, open-source voice assistant inspired by Alexa, documenting the challenges and progress. The project aimed to create a privacy-focused alternative using local models, with ongoing improvements and fixes.
A user is seeking a locally hosted, self-contained app to generate English subtitles (in .srt or .ass format) from video files. They consider Qwen-ASR and Whisper as strong options but report poor subtitle timing in ComfyUI implementations and unreliable performance with older models like those in storytoolkitAI. They ask for recommendations that work well on Windows and can handle multiple languages.
NAR-MBR decoding improves speech recognition by maximizing expected utility from sampled outputs of non-autoregressive models. It achieves better performance than prior NAR methods and runs faster than autoregressive decoding across multiple corpora.
A study finds bilingual fine-tuning enhances automatic speech recognition in low-resource languages when language identification is accurate. Including a language identification token at inference improves ASR performance when identification accuracy is low, especially in diverse language pairs across different families and writing systems.
The wav2vec2.0 model shows no evidence of perceptual compensation for Mandarin tones in embedding similarities. Probing classifiers reveal limited compensation and fail to match human performance on isolated syllables, suggesting supervised training is needed for phonological regularity abstraction.