Language Calibration Impacts Block-Influence Pruning in Multilingual Models
October 16, 2026
Pruning Transformer blocks using Block-Influence (BI) based on English calibration distributions leads to sub-optimal results in Indian languages like Hindi and Tamil. For Qwen2.5-1.5B and Llama-3.2-1B, late-block BI is 20.7–55.5% higher in these languages compared to English, causing incorrect block removal.
HOW THIS AFFECTS YOU
●
researcherYou should use language-specific calibration sets when pruning multilingual models to avoid perplexity damage.