Mechanistic Analysis Reveals World Modeling and Feature Interference in Transformers
September 21, 2026
Analysis of TaxiGPT shows that transformers represent environments, including intersections and streets, using internal maps and goal compasses. Failures in navigation stem from interference between superposed intersection features, which can be mitigated through affordance packing.
HOW THIS AFFECTS YOU
●
researcherYou can apply affordance packing to reduce localization errors in models attempting to build internal world models.