Hugging Face Transformers Adds Native GGUF Support
September 23, 2026
The transformers library now supports loading GGUF files directly using AutoModelForCausalLM. On Apple Silicon, the implementation leverages ggml kernels to achieve performance parity with llama.cpp, reaching 70.4 tok/s for Qwen3.5-4B Q4_K_M on an M2 Max.
HOW THIS AFFECTS YOU
●
builderYou can now use quantized GGUF weights directly within existing PyTorch-based evaluation and inference workflows.