Correctness-Gated Distillation Shows Inconsistent Improvements in Student Models

An investigation into correctness-gated multi-teacher distillation shows that while accuracy and F1 scores may increase, rationale grounding and recall for refuted claims can fail. In some cases, the distillation process led models to assign 'NotEnoughInfo' to all claim examples.