When Prompts Become Pixels: Prompt-Region Grounding for Multimodal Reasoning

Visualized Task Semantics (VTS) identifies a semantic channel gap in MLLMs where accuracy drops by 17.8 points when instructions are embedded as pixels rather than text. Prompt-region grounding mitigates this by aligning question regions with typed semantics, increasing VTS accuracy from 58.0 to 66.3 across four benchmarks at matched training costs.