Semantic Recoverability Determines Downstream Task Success in SSL

This analysis defines semantic recoverability as the factor determining which downstream tasks remain achievable after self-supervised learning. The study demonstrates that recoverability governs directional class-distance-normalized variance and the strength of task-relevant semantic directions in centered and whitened representations.