Towards Audio Token Compression in Large Audio Language Models

Reducing audio token density from 25 tokens/s using unsupervised segmentation and uniform average pooling decreases input sequences by up to 3x. Integrating low-rank adapters during finetuning recovers performance in ASR and speech-to-speech translation tasks, maintaining accuracy near frame-level models while lowering attention complexity for LLM decoders.