Reading Emotions in the Token Space: Discriminative Adaptation of SpeechLLMs for Emotion Recognition

Discriminative adaptation improves speech-based emotion recognition by replacing generative decoding with a single linear classification head applied to the final prompt token's hidden state. This method prevents out-of-set label hallucinations and improves Macro F1 scores on the IEMOCAP dataset across multiple architectures, specifically benefiting realistic ASR transcripts.