DPO-Tuning Causes Emotional Intensity Undershoot in LLMs
September 9, 2026
Direct Preference Optimization (DPO) leads to significant emotional undershooting, with Llama-3.1-8B achieving only 0.26 valence and 0.13 arousal gains relative to requested targets. The phenomenon is attributed to neutral-heavy training corpora and the lack of extreme affect in sampled candidates.
HOW THIS AFFECTS YOU
●
builderYou may need to adjust your fine-tuning datasets if your application requires high-intensity emotional responses.
●
researcherYou can investigate how preference-learning pipelines suppress the distribution of extreme affective states.