F-WANDA: Fisher-Reweighted Post-Training Pruning for LLMs

F-WANDA improves LLM pruning by reallocating the keep budget based on empirical Fisher information from a single backward pass. On Llama-2-7B at 50% sparsity, it achieves a 5-shot MMLU improvement of +1.6 pp over WANDA while using one-third of the energy of SPARSEGPT.