N-Gram Memories at Two Bits: Quantizing Lookup Tables in Large Language Models

Conditional Program (CP) compresses n-gram lookup tables by representing rows via short codes and local scales using a shared Gaussian trellis. This method achieves a 41–63% lower output KL compared to 2-bit round-to-nearest quantization. The architecture enables selective reconstruction of requested rows, maintaining capacity while minimizing memory footprint.