Novelty Patching Increases Byte-Level Model Accuracy via LZ77-Style Compression

A new patching method for Byte Latent Transformers starts patches where upcoming bytes are unlikely to repeat recent context. When trained on a 1.9 GB math corpus, this method increased 50M parameter model final-answer accuracy from 60% to 90%.