CIG-SFT Uses Conditional Information Gain for Better Post-Training

CIG-SFT introduces an information-theoretic framework to optimize supervised fine-tuning. It distinguishes between tokens that conflict with pretraining and those that provide necessary task-specific instruction by estimating task-conditioned information gain.