SFT Induces Diversity Collapse in LLM Sequential Decision-Making

Supervised fine-tuning on single-action demonstrations causes premature diversity collapse in board game environments without a proportional gain in accuracy. Implementing action augmentation, which trains on all optimal actions per state, can partially mitigate this narrowing effect.