TiCodec proposes a factorized representation that separates time-varying speech content from time-invariant information using a Time-Invariant Representation Extraction (TIRE) module. This approach aims to reduce the amount of information that must be modeled at the frame-level, facilitating low-latency speech processing.
- Probing tasks reveal that intermediate encoder layers capture complementary speaker- and environment-related information while containing little linguistic content.
- Cross-file sampling during TIRE training improves the robustness of invariant representations.
- The proposed Dual-TIRE architecture exploits this complementarity to improve speech reconstruction quality and speaker similarity.
- Streaming inference using successive 660ms processing blocks achieves operation without significant degradation in reconstruction performance.
The results highlight the potential of factorized neural codec representations for future low-latency speech generation systems.