TTPO Enables Test-Time Policy Optimization for LLM Reasoning
August 28, 2026
TTPO uses an asymmetric objective to improve mathematical reasoning by distilling agreeing rollouts via On-Policy Self-Distillation and penalizing disagreeing rollouts with Grouped RL. This approach overcomes the fragility of majority-vote pseudo-labels during test-time training.
HOW THIS AFFECTS YOU
●
builderThis provides a method to improve model reasoning performance during inference-time compute.
●
researcherYou can optimize policies at test-time without relying on ground-truth labels.