Researchers introduce SPEARBench, a benchmark designed to evaluate the naturalness of streaming speech-to-speech language models during question-answer interactions.
- The benchmark constructs controlled dialogue prompts from the Seamless Interaction corpus and runs inference across multiple models.
- It utilizes a multidimensional evaluation protocol covering response latency, interruptions, speech quality, ASR robustness, language and dialect consistency, emotional naturalness, interpersonal stance, and explainable distributional baselines.
- Original human answers serve as a reference condition for comparison.
Results indicate that while current models achieve high signal-level quality and low ASR error, they still differ from human conversational behavior in latency, overlap, dialect preservation, emotional adaptation, and interpersonal stance dynamics.