Personalizing MoE LLMs via Interpretable Latents and Crosscoders
October 16, 2026
Personalizing Mixture-of-Experts (MoE) models reduces polysemanticity compared to dense models by routing tokens to specialized experts. Using crosscoders on paired activations, this method identifies specific experts encoding distinct personalization features while maintaining competitive performance.
HOW THIS AFFECTS YOU
●
builderYou can implement more efficient, user-centric personalization that activates only a fraction of total parameters.
●
researcherYou can achieve higher interpretability in personalized models by localizing features to specific experts.