Intelligence per Watt: Assessing Local AI Inference Efficiency

This study evaluates the viability of local inference using models under 20B parameters on accelerators like Apple M4 Max. It argues that the rise of small, capable models creates a shift where local hardware can compete with centralized cloud infrastructure on energy efficiency.