A new affine translator can replace up to 96% of transformer blocks in Tabular Foundation Models (TFMs) without sacrificing performance. Testing across 171 datasets reveals that current TFMs are significantly overparameterized for tabular classification tasks compared to shallower architectures.
HOW THIS AFFECTS YOU
●
builderYou can significantly reduce the computational and memory footprint of tabular models in production.
●
researcherYou can use this method to study the architectural redundancies inherent in tabular transformers.