ELBO-Based Likelihood Estimation Improves RL for Diffusion Models

Systematic analysis of RL for diffusion models identifies the Evidence Lower Bound (ELBO) likelihood estimator as the primary driver for efficient training. The study disentangles policy-gradient objectives, estimators, and rollout schemes to show that computing likelihood from the final sample enables better performance.