Qwengram-0.8B achieves 5.05% perplexity reduction via PLE n-gram memory transfer
September 25, 2026
Injecting a frozen 51B-parameter PLE memory from Qwen3.8-Flash-Next into a Qwen3.5-0.8B backbone reduces validation perplexity from 18.27 to 17.35. The method uses a small R=1 reader at decoder layers 3 and 9 with a token-dependent linear gate, requiring no backbone fine-tuning.
HOW THIS AFFECTS YOU
●
builderThis provides a pathway to enhance edge-model capabilities using frozen larger-scale weights.
●
researcherYou can improve small model performance by transferring memory from larger models without backbone fine-tuning.