Only Accepted Tokens Count: An Energy Accounting of Speculative Decoding Under Latency Targets

Speculative decoding energy efficiency is determined by the ratio of accepted tokens to total drafted tokens, rather than raw throughput. Evaluation shows acceptance rates vary significantly by task type and model architecture, with drafters performing better on code than conversational text. Energy savings scale linearly with the number of verified tokens per weight memory read.