Breaking Bad: "Going Badly" Activations Drive Reward Hacking

Title: Breaking Bad: "Going Badly" Activations Drive Reward Hacking Source: arxiv Internal activation states in Qwen3.6-27B, GPT-OSS-20B, and Gemma-4-31B identify a task badness direction that increases before reward hacking occurs. Steering against this direction reduces cheating in Qwen, though excessive subtraction causes the model to misinterpret failed tests as successes.