SAGE: Surrogate-gradient Adaptation via Attention-Guided Entropy for Spiking Transformers
August 17, 2026
SAGE adapts surrogate-gradient slopes in Spiking Transformer training by using normalized self-attention entropy to estimate block-level uncertainty. This mechanism modulates only training-time parameters to improve optimization without altering the inference architecture or deployment energy efficiency. Experiments on CIFAR-10/100 demonstrate improved accuracy over fixed-surrogate baseline methods.