Adapting English Quality Classifiers for Multilingual Pretraining Data

This method uses a multi-layer perceptron on top of Transformer embeddings to adapt English quality classifiers for over 100 languages. It utilizes machine-translated English labels to select high-quality training data for low-resource languages in 1B, 3B, and 8B scale experiments.