TreeSpark Load-Adaptive Draft Trees for Speculative Decoding
September 22, 2026
TreeSpark improves speculative decoding by using parent-conditioned distributions from a drafter's Markov head to calibrate edge-acceptance estimates. It implements best-first tree expansion and load-adaptive stopping to optimize verification efficiency based on current serving load.
HOW THIS AFFECTS YOU
●
builderYou can implement more efficient semi-autoregressive inference that adapts to real-time serving loads.
●
researcherThe use of parent-conditioned distributions provides a more mathematically sound way to construct draft trees.