Multi-Teacher Distillation for Unified Speech Attribute Representations

A multi-teacher framework uses a shared SSL backbone and attribute-specific branches to simultaneously learn semantic, speaker, language, and emotion representations. The approach achieves performance near single-task models while utilizing only one shared encoder.