Probing Audio-Language Models as Judges for Generated Music
October 12, 2026
Four probes applied to MusicEval and SongEval reveal that audio-language judges struggle to outperform prompt-only predictions when audio is removed. The study shows that while models follow audio cues for alignment, they fail to match expert human rating correlations in key areas.
HOW THIS AFFECTS YOU
●
researcherYou should be cautious when using audio-language models as automated evaluators for music generation.
●
designerThis highlights the current limitations in using AI to provide nuanced, human-like aesthetic feedback on creative content.