Analyzing Prediction Dynamics in Depth-Recurrent Language Models

This study decomposes prediction refinement in depth-recurrent models like Huginn-3.5B and Ouro-1.4B. By accounting for update direction and competitor pairing, researchers reduced the mean earliest qualifying depth by 22.5-34.4% compared to standard translation removal.