Microsoft's NextLat Gets 3.3x Inference Speedup via Latent Self-Prediction

Microsoft Research's Next-Latent Prediction trains transformers to predict their own next latent state alongside next-token prediction, enabling self-speculative decoding with up to 3.3x inference speedup. The method also improves data efficiency by providing denser supervision signal in latent space compared to one-hot token targets.