Collaborative Learning Replaces Large Models with Heterogeneous Model Slices

Model slicing replaces a single large model with smaller, heterogeneous models under a comparable parameter budget to reduce error propagation. The approach uses pyramid supervision and protocol distillation to enable effective cross-model collaboration and reassess prior reasoning steps.