Physical prompt injection attacks succeed on 27-29% of VLM-controlled robots
August 7, 2026
Adversarial text placed in a robot's visual field can trigger indirect prompt injections in VLM planners. In a study of GPT-4o, Gemini 2.5 Flash, and Qwen3-VL-32B, physical text attacks successfully hijacked robot tasks in nearly 30% of trials.
HOW THIS AFFECTS YOU
●
builderYou must implement visual safety guardrails to prevent text in the environment from overriding commands.
●
policyThis highlights a critical new physical attack surface for autonomous systems using VLMs.