Mitigating Simulator Collapse in Multi-Agent Reinforcement Learning

Training LLM policies against a single simulator LLM leads to simulator collapse, where policies overfit to the simulator's limited behavioral modes. The paper introduces Verbalized Sampling for inference-time behavior broadening and Co-Training to optimize policies against a population of trainable simulators.