Simplifying Neural Networks via Neural Collapse and Inverse Fisher Criterion

This framework simplifies overparameterized networks by monitoring representation dynamics through the Inverse Fisher Criterion. It identifies the split between feature extraction and classification to replace trailing layers with lightweight heads during training.