Post-training methods amplify demographic opinion imbalances in LLMs
October 27, 2026
Controlled experiments across SFT, DPO, and GRPO show that post-training paths amplify existing opinion imbalances in training corpora. The ability to remove these amplified preferences through subsequent training depends heavily on the specific post-training method and the initial mixture of opinions.
HOW THIS AFFECTS YOU
●
researcherYou can use these findings to study how different alignment algorithms interact with data distributions.
●
policyThis highlights the difficulty of neutralizing bias once it is baked in during alignment stages.