[arXiv]score: 0.49
FRONT Uses DCT Low-Frequency Weights as Architecture-Agnostic Knowledge Transfer for Model Init
May 26, 2026
FRONT isolates task-agnostic 'learngene' knowledge in the low-frequency DCT components of pretrained model weights, enabling initialization of downstream models with different architectures without requiring access to large model collections.
cs.LG
HOW THIS AFFECTS YOU
●
builderYou can use FRONT to initialize models of varying scales from a single pretrained source without parameter selection heuristics or generative model dependencies.
●
researcherEmpirically demonstrates that low-frequency weight components encode transferable, architecture-agnostic knowledge, providing a new lens on what pretrained models actually learn.