SAE Analysis Suggests RL-Trained Models Rely on Prompt Patterns over Reasoning
October 30, 2026
Mechanistic interpretability using sparse autoencoders (SAEs) on models trained with GRPO in a Tic-Tac-Toe domain reveals that RL gains may stem from adapting to explicit prompt information rather than learning high-level logical concepts. While accuracy increases, internal representations primarily track board occupancy and turn order.
HOW THIS AFFECTS YOU
●
researcherYou should investigate whether RL-driven performance gains in your models are genuine reasoning or pattern exploitation through SAE-based probing.