SiPE: Syntax-Informed Positional Embeddings for Transformers

Syntax-Informed Positional Embeddings (SiPE) injects lightweight dependency parse priors into absolute, relative, or rotary positional embeddings. For autoregressive decoders using relative PE, multiplicative injection into the attention score provides the strongest performance gains without altering the core architecture.