Adversarial Decoys Bypass Attention-Based Defenses in Vision Transformers

Adversarial decoys exploit the coupling between attention scores and defense mechanisms in Vision Transformers (ViTs). This method uses independently optimized patches to redirect attention rankings, effectively neutralizing test-time defenses that suppress high-attention adversarial tokens.