RheoSampling Decouples Probability Roles in Speculative Decoding
September 21, 2026
RheoSampling resolves the stochastic decoding dilemma in dynamic-tree speculative decoding by decoupling the roles of tree construction and token verification. This prevents the draft distribution from collapsing into one-hot probabilities, maintaining higher acceptance rates during stochastic sampling.
HOW THIS AFFECTS YOU
●
builderThis technique could lead to faster LLM inference while maintaining the quality of stochastic sampling.
●
researcherThis addresses a fundamental coupling issue between tree topology and sampling distributions in speculative decoding.