Small VLMs Require Fine-Tuning to Match Human Cross-Modal Choice Alignment

While larger vision-language models show some alignment with human cross-modal associations, their visual saliency often fails to match human gaze patterns. Fine-tuning smaller VLMs on human choice data can bring their decision alignment to human majority-vote levels, though visual attention remains distinct.