Liminal Training Mitigates Subliminal Trait Transfer in Knowledge Distillation
September 22, 2026
Subliminal learning allows unintended traits to transfer from teacher to student models during distillation. The authors introduce liminal training, an annealed KL-regularized fine-tuning method that reduces this non-monotonic trait acquisition in models ranging from 1.5B to 8B parameters.
HOW THIS AFFECTS YOU
●
builderYou can implement liminal training to prevent your distilled student models from inheriting unwanted teacher biases.
●
researcherThis provides a new method for controlling unintended behavioral drift during distillation.