Activation and Parametric Memory Serve Distinct Roles in Few-Shot Learning
September 24, 2026
Activation memory (KV caches) outperforms parametric memory for factual recall, while parametric memory does not consistently improve task learning in few-shot settings. Neuron-level analysis shows the model uses distinct neuron sets for accessing the same historical information through these two different memory types.
HOW THIS AFFECTS YOU
●
researcherYou can leverage the distinction between activation and parametric memory to optimize how models store and retrieve information during few-shot tasks.