Game-Theoretic Foundations for Undiscounted Bisimulation Metrics in RL
October 30, 2026
This framework extends bisimulation metrics to undiscounted, total-reward, and average-reward MDP settings by treating metric operators as Bellman-Shapley operators for zero-sum games. This approach provides unique metric definitions even when the discount factor equals one.
HOW THIS AFFECTS YOU
●
researcherThis allows you to apply bisimulation-based state similarity to a broader class of reinforcement learning problems.