WhiteMatter introduces a router-based architecture that allows every attention layer to access mixed KV channels from all past token layers, potentially reducing KV-cache memory footprints.
HOW THIS AFFECTS YOU
●
builderThe ability to control KV-cache size through $k$ channels may improve deployment efficiency.
●
researcherThis method offers a new way to implement feedback-style connections in Transformers.