Structural Equivalence and Learning Dynamics in Delayed MARL

Researchers at arXiv formally prove structural equivalence between Observation Delay and Action Delay in Dec-POMDPs, showing both yield identical admissible joint-policy sets and trajectory distributions, generalizing single-agent infinite-horizon results to any-horizon cooperative multi-agent settings. Mixed-delay configurations reduce to pure OD systems, and in Transition-Independent MDPs, histories compress to tractable local augmented states. Multi-agent RL practitioners designing delayed communication systems or sim-to-real pipelines should care, as this eliminates redundant algorithmic development across delay types while exposing critical learning dynamics differences despite equivalent optimal solution spaces.