Centralized Critics for N-Agent Ad Hoc Teamwork Coordination

This research compares controlled history estimators and joint history estimators for centralized training in multi-agent reinforcement learning. It specifically examines how to design critics for ad hoc teamwork where agents must coordinate with unknown, uncontrolled teammates.