Disentangling Language Modeling and Byte-Level Boundaries

This research proposes that byte-level language models allow for the independent manipulation of next-byte prediction and patch boundary placement. This structural advantage could enable exact knowledge transfer between models regardless of their original tokenization.