Minimizing Off-Policy Evaluation Variance via Budgeted Multi-Source Annotations
October 9, 2026
This method formulates an integer allocation problem to select the most cost-effective counterfactual annotations from sources like LLMs or human experts. It uses a majorization-minimization algorithm with dynamic programming to minimize estimator variance in contextual-bandit off-policy evaluation.
HOW THIS AFFECTS YOU
●
builderThis provides a framework for managing the cost of acquiring ground-truth data for reinforcement learning systems.
●
researcherYou can optimize the selection of noisy or costly annotation sources to improve OPE stability.