NCP-ArchPreview: 8.9B Parameter Latent-Space Model using Next Concept Prediction
September 11, 2026
NCP-ArchPreview scales to 8.9B parameters trained on 5.73T tokens using a joint Next-Token Prediction and Next-Concept Prediction objective. The model constructs a product-quantized concept vocabulary from hidden states to enable autoregressive generation guided by discrete, multi-token concepts.
HOW THIS AFFECTS YOU
●
researcherYou can move beyond standard token-level autoregression by integrating quantized concept-level objectives into pretraining.
●
founderThis suggests a shift toward architectures that process semantic concepts rather than just sub-word tokens.