Experience Distillation Retains 64.8% of In-Context Learning Gains
July 24, 2026
Experience Distillation internalizes agent interaction histories into model weights without requiring additional environment samples. On 749 software engineering tasks and six text adventures, the method retains at least 64.8% of in-context learning performance compared to standard supervised fine-tuning.
HOW THIS AFFECTS YOU
●
builderYou can improve agent performance using past interaction logs without the high cost of new environment trials.
●
researcherYou can bridge the gap between sample-efficient in-context learning and weight-based model intelligence.