Power Law Graph Attention replaces scaled dot-product attention
August 9, 2026
Power Law Graph Attention (PLGA) replaces the fixed bilinear form of scaled dot-product attention with an input-generated bilinear operator using elementwise power laws. While the architecture generalizes SDPA, empirical results show attention collapse during inference.
HOW THIS AFFECTS YOU
●
researcherYou can explore power-law-based attention mechanisms, though you must account for empirical collapse at inference time.