Efficient LLM Distillation via Offline Top-K Logits

Offline knowledge distillation using cached teacher top-K logits matches online performance while running 29% faster per iteration. This method achieves up to 41% higher throughput on a single H200 GPU by removing the teacher from the training memory loop.