Smarter Pair Selection in DPO Training Outperforms Random Comparison Sampling

Framing comparison curation in DPO as a sampling-design problem, the paper shows that labeling the most informative pairs from a larger completion pool yields better policy performance than labeling all pairs from a smaller pool under the same budget. The framework analytically traces how pair choice propagates through DPO to downstream policy quality.