CAFIN Accelerates AdaFisher via CUDA Kernel Fusion
November 13, 2026
CAFIN implements GPU-native kernel fusion for the AdaFisher second-order optimizer. By integrating FIM construction, EMA updates, and gradient preconditioning into fused kernels, it reduces memory traffic and sequential CUDA kernel launch overhead.
HOW THIS AFFECTS YOU
●
builderThis provides a faster way to implement advanced optimizers in custom training pipelines.
●
researcherYou can use second-order optimization more efficiently by reducing the overhead of Fisher Information Matrix approximations.