Optimal CNN Weights for Contrastive Learning via Sinusoid Filters

Analytic computation shows that the optimal representation for contrastive loss using basic augmentations can be achieved by a CNN featuring sinusoidal first-layer filters and partial whitening. The filter frequencies and weights are computable using a waterfilling algorithm based on the dataset's power spectrum.