Analyzing FLOP Increases in Speculative Decoding Efficiency

Speculative decoding improves inference speed not by reducing total FLOPs, but by shifting GPU workloads toward more efficient parallel verification. While the total floating point operations increase, the technique leverages parallel execution of draft tokens to optimize memory bandwidth utilization.