LLMs Formally Modeled as Monotone Compression Algorithms

This theoretical study proves that next-token predictors are equivalent to monotone, or order-preserving, compression algorithms. The work establishes a mathematical link between language modeling objectives and the capacity for efficient information compression.