When does text encoding matter for language modeling?

UTF-8's variable-length design creates cross-lingual performance inequities in byte-level language models. This study evaluates UTF-8 against Unicode-based and language-specific strategies, introducing Aether, a hybrid scheme that applies script-specific encodings with UTF-8 fallbacks. Findings indicate that information-dense scripts like Chinese and Korean derive significant benefits from compressive encodings.