Routing Entropy Fails as an Independent Uncertainty Signal in Transformers
October 2, 2026
Audits of Attention-Residual Swin-Tiny and DeiT-Small models show that routing entropy does not provide corrective information beyond the model's own confidence scores. Statistical testing indicates that routing traces do not reliably signal correctness even when using soft-binned calibration auxiliary losses.
HOW THIS AFFECTS YOU
●
researcherYou cannot rely on routing traces as a proxy for uncertainty in dynamic architectures.