Locating Reasoning Subspaces in Distilled Student Models

Research into distilled student models shows that they retain 'thinking-like' behavior in specific adapter subspaces. By identifying and removing these subspaces, researchers can effectively turn off reflective markers and monologue formats in smaller models.