Mapping English Text to Hindi Speech via Visual Grounding
August 28, 2026
An alignment-based approach uses self-supervised speech representations and off-the-shelf image captioning to map written English keywords to spoken Hindi. This method avoids end-to-end multimodal training by aggregating alignment evidence from images with Hindi spoken captions.
HOW THIS AFFECTS YOU
●
researcherThis provides a simpler alternative to end-to-end multimodal models for low-resource speech-text alignment tasks.