Progressive Cramming Reveals Limits of Token Compression

Progressive cramming expands target prefixes token-by-token to identify the exact point where reconstruction fails within a fixed optimization budget. The method shows that even with high reconstruction accuracy, crammed embeddings cause significant accuracy drops in multiple-choice tasks and nearly total collapse in generative capabilities.