Tradeoffs in Speculative Diffusion for LLM Acceleration

Speculative diffusion explores using diffusion models as draft models in speculative decoding to accelerate autoregressive LLMs. Unlike traditional lossless speculative decoding, this approach accepts a quality-speed tradeoff by allowing the diffusion model to generate tokens in parallel.