Effective Attention Set Size Limits LLM Retrieval Capacity

Estimating the number of tokens required to maintain NLL close to full-attention baselines reveals that relatively small, high-weight sets are sufficient. The study shows that attention-based selection significantly outperforms random token selection for retrieval.