Mathematical Analysis of the Jacobian Lens for Model Interpretability

This study characterizes the Jacobian lens as a first-order causal transfer operator that approximates the mapping from intermediate activations to future readouts. Analysis reveals that the causal geometry of the Jacobian is highly sparse, with energy concentrating in a small proportion of critical positions and decaying with depth.