Mixture-of-Translators Enables KV Cache Sharing Across Different LLMs

Mixture-of-Translators (MoT) maps key-value (KV) caches from a source LLM to a target LLM's cache space using multiple translator modules. This framework allows heterogeneous models to reuse shared contexts, reducing the need for redundant prefilling during multi-model reasoning.