Free Pause Tokens for Parallel Compute in Transformers
September 4, 2026
A new method for adding thinking-style compute to language models via a parallel prediction stream over a weight-shared backbone. It achieves a 2-3 centinat improvement in next-token prediction on a 1B parameter model without increasing context length, KV cache, or inference latency.
HOW THIS AFFECTS YOU
●
builderYou can potentially gain reasoning capabilities without increasing your inference cost or latency.
●
researcherYou can study how parallel streams avoid sequence length penalties.