Relative Value Learning learns value differences via pairwise Bellman operators

Relative Value Learning (RV) replaces absolute state value estimation with an antisymmetric function that learns value differences directly. Integrated with PPO, the method achieves competitive results on 49 Atari games using a new unbiased policy-gradient estimator, R-GAE.