Training Transformers from Scratch on $8 ESP32-S3 Microcontrollers
August 5, 2026
The Qapla' project demonstrates end-to-end transformer training, including forward passes, backpropagation, and weight updates, directly on an ESP32-S3. This moves beyond simple inference to enable training small language models on low-cost, $8 edge hardware.
HOW THIS AFFECTS YOU
●
builderYou can implement local, offline model training on extremely cheap edge devices.
●
researcherYou can explore ultra-low-power training architectures and hardware-constrained optimization.