TopicScope uses SAE features to monitor changing topics in Gemma-3
November 11, 2026
TopicScope is a framework that trains causal topic classifiers on sparse autoencoder features extracted from prefill hidden states. Monitors trained on Gemma-3 1B and 4B prefill states successfully generalize to decoding, allowing for topic labels that change partway through a response.
HOW THIS AFFECTS YOU
●
builderYou can implement monitors that track changing topics within a single model response.
●
researcherYou can use SAE features from prefill states to build monitors that work during decoding.