Free Energy-Driven Reinforcement Learning with Adaptive Advantage Shaping for Unsupervised Reasoning in LLMs

FREIA, an unsupervised RL algorithm for LLMs, uses Free Energy-Driven Rewards to balance consensus-exploration tradeoffs and Adaptive Advantage Shaping to dynamically adjust learning signals based on reward statistics, achieving improvements over unsupervised baselines across nine datasets spanning mathematical reasoning, commonsense, and logical inference tasks.