ReRULE Off-Policy Replay Improves LLM Unlearning Efficiency via Hard-Case Reuse

ReRULE augments GRPO-based reinforcement unlearning by storing low-reward hard-case rollouts in a replay buffer and reusing them via importance-sampled off-policy updates, targeting the boundary between forget and retain sets. The approach addresses the gradient starvation problem where easy cases converge quickly and hard cases are discarded after a single use.