COFFEQA Method for Counterfactual-Guided Few-Shot Knowledge Distillation
October 12, 2026
COFFEQA improves LLM question-answering distillation by using counterfactuals to provide supervision around teacher decision boundaries. This strategy avoids the performance degradation typically seen when using high-uncertainty samples during few-shot distillation.
HOW THIS AFFECTS YOU
●
builderYou can use this approach to train smaller, resource-efficient models more effectively with fewer training samples.
●
researcherThis addresses the instability of few-shot distillation caused by sampling near decision boundaries.