Expander SAEs Reduce Sparse Autoencoder Decoder Costs via Expander Masks

Expander SAEs utilize a left-d-regular expander mask to support TopK sparse autoencoders, reducing decoder learning from O(mn) to O(dn) parameters. This structure converts the matching-pursuit correlation step into an efficient gather-and-reduce operation while maintaining feature sparsity.