Retrieval heads in MLLMs extend across audio-visual modalities

Analysis shows that retrieval heads—sparse attention heads used for locating evidence—remain sparse and functional when transitioning from text to audio-visual modalities. This suggests the mechanism for evidence localization is a generalizable property of MLLMs.