Geometric Stratification of Transformer Residual Streams

Investigation of 18 models from 7B to 120B parameters reveals that the prediction direction acts as a content-defined privileged anchor within residual streams. This creates a geometrically stratified structure where readout-relevant information concentrates in a narrow, scale-invariant prediction interface.