Predicting LLM Reasoning Correctness via Information Density Metrics

Information density in chain-of-thought (CoT) steps, measured by LP variance and Caveman compression ratios, can predict answer correctness. The relationship between density and accuracy varies significantly depending on whether the model was fine-tuned via SFT or RL distillation.