A new readout phase for ResonatorLM makes the memory read a function of the current token. This adds input dependence to the readout while maintaining O(T log T) complexity and adding only 0.2% more parameters.
HOW THIS AFFECTS YOU
●
researcherYou can achieve content-dependent memory in sequence models without the quadratic cost of attention.