Envelope Sampling Mitigates Reward Hacking in Surrogate Post-Training
October 9, 2026
Envelope sampling provides a theoretically-grounded method for recalibrating LLM judges using a small number of ground-truth human annotations. This approach minimizes the upper bound on post-trained model regret, addressing reward hacking caused by miscalibrated surrogates.
HOW THIS AFFECTS YOU
●
builderThis offers a path to use cheaper LLM judges for RLHF while reducing undesirable side effects from reward hacking.
●
researcherYou can use this method to theoretically bound regret during judge recalibration.