Surfing the Activation Wave: Analyzing the Information and Geometry of LLM Hidden States for Speculative Decoding

A framework for speculative decoding selects optimal LLM layer subsets using matrix-entropy to measure informativeness and centered kernel alignment (CKA) to ensure feature diversity. This replaces fixed, position-based heuristics with a principled method to capture complementary information across hidden states, addressing the combinatorial complexity of layer selection.