SpecOPD: Parallelized On-Policy Distillation for Speculative Decoding
November 1, 2026
SpecOPD accelerates speculative decoding by parallelizing draft sampling and target supervision across independent branches. This framework avoids the sequential dependencies of conventional on-policy distillation, pushing drafter performance beyond standard acceptance-length plateaus.
HOW THIS AFFECTS YOU
●
builderYou can increase LLM inference throughput by implementing parallelized on-policy distillation for drafters.