Bellman-Consistent Trajectory RL for LLM Arithmetic Reasoning

The TBRM-RLOO framework converts sparse rewards into dense token-level signals to stabilize arithmetic reasoning training. It improves Pass@1 to 0.64 and achieves a state-of-the-art Pass@16 of 84.0% on the Countdown task, outperforming DPO and GRPO.