[arXiv]score: 0.13
Canopy Entropy Metric Reveals Fine-Tuning Increases Information Efficiency
June 1, 2026
Canopy Entropy (CE*) measures generation-space size by treating rollouts as a tree, jointly capturing length and sequence uncertainty as total Shannon entropy H(N, Y|X). Analysis shows fine-tuning improves information conveyance efficiency rather than simply reducing diversity, with output length identified as a key confounder in prior work.
cs.CLcs.AIstat.ML
HOW THIS AFFECTS YOU
●
researcherCE* offers a theoretically grounded alternative to token-level entropy for evaluating fine-tuning effects — the length-entropy correlation term provides a new diagnostic for generation quality.