Distributionally Robust BEST-RQ for Speech Representation Learning

A new extension to the BEST-RQ self-supervised learning method uses group Distributionally Robust Optimization to create speech representations that are less sensitive to quantization targets. The approach adaptively reweights poorly performing groups to improve robustness across multiple configurations.