BranchOPD improves on-policy distillation by selecting decision points based on student uncertainty and teacher disagreement. By reusing shared prefixes for branching trajectories, it reduces student-generated token costs by 19% on ALFWorld and 31% on WebShop.
HOW THIS AFFECTS YOU
●
builderYou can significantly lower the compute and token costs of distilling student models for agentic tasks.
●
researcherThis approach provides a more efficient way to increase supervision coverage under finite sampling budgets.