Learned activation patching methods often report higher effect sizes than whole-state copying because optimization objectives select components that trigger compensatory output movements. This mathematical bias causes researchers to overestimate how cleanly a learned part isolates a concept compared to the full model state.
HOW THIS AFFECTS YOU
●
researcherYou should account for optimization bias when interpreting activation patching results as evidence of concept isolation.