DMAPO Achieves Higher Alignment with 3.45% of Data
July 29, 2026
The DMAPO method uses multi-evaluator agreement to select high-confidence preference optimization data, retaining only 3.45% of Mistral-7B candidates. KTO training on this distilled set achieved a 7.50 MT-Bench score and a 95.5% win rate against text-davinci-003 in length-controlled evaluations.
HOW THIS AFFECTS YOU
●
builderThis demonstrates that extremely high-quality, small datasets can outperform much larger, noisier preference datasets.
●
researcherYou can improve alignment efficiency by focusing on high-consensus, rubric-specialized evaluator data.