Many-body Tipping Dynamics Explains Predictable Failure Modes in LLMs

This study identifies LLM failures, such as harmful or repetitive outputs, as many-body dynamical processes between tokens crossing finite layers. By modeling these as first passage processes between competing output basins, the research suggests that attention disorder can be used to predict model tipping points.