QLPO for Length-Aware Policy Optimization in Reasoning Models

QLPO is a resampling-based variant of GRPO that achieves implicit length control in reasoning models by favoring short correct responses during training. Testing on models from 1.5B to 32B parameters shows it reduces inference latency and cost without requiring explicit length penalties or reward function modifications.