[HN]score: 0.06
PyTorch-native TPU back end for vLLM
October 19, 2026
vllm-torchtpu provides a PyTorch-native backend for vLLM that runs standard model definitions on Google Cloud TPUs via torch.compile. It uses Pallas-based kernels for attention, MoE, and FP8/INT8/MXFP4 quantization while supporting tensor, data, pipeline, and expert parallelism. The plugin enables TPU-speed serving without requiring JAX re-implementations.
DAILY DIGEST
you don't check 9 sources — we do. one email every morning, read in 2 min. free. unsubscribe anytime. privacy