Natural Language Autoencoders Generalize for LLM Activation Verbalization
October 23, 2026
Natural language autoencoders (NLAs) translate internal LLM representations into human-readable text by reconstructing activations through a natural language bottleneck. Experiments show NLAs can effectively generalize to finetuned models and recover finetuning objectives during model diffing tasks.
HOW THIS AFFECTS YOU
●
researcherYou can use NLAs to inspect internal model state changes and understand compositional structures in activations.