Visual Content Scales Multimodal Loss via Power Law

Research shows that including visual content in native multimodal pre-training improves multimodal loss following a power law relative to compute, without affecting baseline text-only performance. This suggests vision acts as a scalable anchor during training.