Uncheatable Eval Uses Dynamic Compression to Combat Data Contamination

Uncheatable Eval uses lossless data compression rates on newly published text to evaluate base language models and mitigate benchmark contamination. The method correlates compression performance with zero-shot MMLU accuracy across 80 models and 14 text categories.