Revisiting Lossy Verification in Speculative Decoding: Mechanisms, Trade-offs, and Failure Modes

Lossy verification in speculative decoding introduces distributional shifts that degrade generation quality by relaxing strict matching requirements. Analysis classifies existing methods into truncation-based or collaborative verification schemes, identifying a fundamental risk where truncation-based approaches cause silent performance degradation. A new diagnostic framework evaluates these stability trade-offs across multiple benchmarks.