LLM Embeddings Exhibit Circulant Gram Matrix and Circular Geometry
September 26, 2026
New research shows that LLM embeddings project into a circle in high-dimensional space and exhibit an approximately circulant Gram matrix. This finding applies even to low-dimensional word2vec embeddings and provides an analytical framework for embedding structures.
HOW THIS AFFECTS YOU
●
researcherYou can use this geometric theory to better understand and manipulate the structure of latent spaces.