Proxy-KD Method Transfers Black-Box LLM Knowledge Without Internal State Access

Proxy-KD uses an intermediate proxy model to bridge knowledge distillation from black-box LLMs like GPT-4 to smaller student models, bypassing the need for logit-level access. The method outperforms standard output-based KD approaches where only teacher completions are available.