CM-DPO Improves LLM Planning via Constraint-Aware Preference Optimization

CM-DPO replaces binary preference signals with continuous margins scaled by violation severity using a symbolic verifier. This method separates hard and soft constraints through a lexicographic objective, allowing an 8B model to achieve superior planning performance on benchmarks like TravelPlanner.