VLMs Conflate Vertical Image Position with Depth, Bias Worsens with Scale

A probing study using minimal contrastive pairs finds that VLMs consistently entangle vertical image position with perceived distance — a perspective bias from natural photo statistics. This accuracy gap between perspective-consistent and counter-heuristic examples grows under data scaling even as overall benchmark scores improve, suggesting benchmarks mask a structural spatial reasoning failure.