HERD Framework Combines Harness-Based Skills with Recursive Distillation
October 23, 2026
HERD uses a hybrid continual learning approach that captures patterns in a non-parametric harness while performing drift-constrained distillation into model weights. This dual mechanism aims to mitigate catastrophic forgetting and plasticity loss by retaining high-residual examples in bounded memory.
HOW THIS AFFECTS YOU
●
researcherYou can explore this method for building models that balance in-context learning with parameter updates.