WMIP Probe Reveals LLM Cumulative State Tracking Limits
October 29, 2026
A working-memory-inspired probe shows that 18 LLMs achieve 84.9% accuracy on single-step arithmetic but drop to 31.7% accuracy after three to seven updates. No model exceeds 10% accuracy when tracking more than 50 updates without writing intermediate steps.
HOW THIS AFFECTS YOU
●
builderYou can improve model reasoning by instructing it to write out intermediate steps for state tracking.
●
researcherThis benchmark provides a way to measure the limits of LLM working memory for cumulative updates.