Beyond Similarity: Temporal Operator Attention for Time Series Analysis

Researchers propose Temporal Operator Attention (TOA), reframing why MLPs/linear models beat Transformers on time-series: standard attention's convex-combination outputs can't represent signed/oscillatory transformations needed for filtering and harmonic structure. TOA reformulates attention as a learned temporal operator, directly addressing this structural mismatch. Practitioners building forecasting pipelines should evaluate this as a principled Transformer fix rather than defaulting to linear baselines.