TreeGraft Uses Multi-Drafter Grafting for Speculative Decoding Acceleration
August 28, 2026
TreeGraft addresses the latency-quality trade-off in tree-structured speculative decoding by using multiple drafters of varying costs. A stronger drafter rescores candidates and selects grafting positions to recover promising paths identified by smaller, faster drafters.
HOW THIS AFFECTS YOU
●
builderYou can reduce inference latency while maintaining higher acceptance rates in speculative decoding.
●
researcherThis introduces a more efficient way to manage multi-path drafting in LLM inference.