CircuitLens uses attention head engagement for RL data selection
September 9, 2026
The Circuit Reasoning Score (CRS) identifies 46 reasoning-sensitive attention heads via contrastive ablation to select training data. On Qwen2.5-Math-7B, selecting the lowest-engagement decile for RLVR improves GSM8K by 2.0 pp and Minerva by 2.9 pp compared to random selection.
HOW THIS AFFECTS YOU
●
builderYou can improve RL training efficiency by filtering data based on frozen model attention patterns.
●
researcherYou can use circuit-based signals instead of reward labels to identify high-value training data.