Tokenization and Encoding Biases in Crosslingual LLM Evaluation

An empirical study reveals that widely used normalized metrics in crosslingual evaluation introduce significant biases due to tokenization, encoding, and orthographic differences. The research suggests sentence-level negative log-likelihood over semantically equivalent pairs as a more robust alternative.