Streaming Neural Speech Codecs through Time-Invariant Representations

TiCodec factorizes neural speech codecs into time-varying content and time-invariant representations using a Time-Invariant Representation Extraction module. Probing tasks show intermediate encoder layers capture speaker and environmental features rather than linguistic content. Cross-file sampling during training improves the robustness of these invariant representations for low-latency processing.