Contrastive Energy Fields for Inference-Time Procedure Planning in Instructional Videos

CEFITO formulates procedure planning as a task-constrained optimization problem to enforce logical constraints during inference. By learning an action-conditioned representation space, the method omits irrelevant actions from the search space rather than treating all actions as equally plausible. This approach achieves state-of-the-art accuracy on two benchmark datasets.