This method installs recurrent depth into a pretrained Qwen2.5-0.5B-Instruct model using a weight-tied Recurrent Block. An adapter using 6M trained parameters achieved 83.8% accuracy on ARC benchmarks, nearly matching a full 180M parameter block.
HOW THIS AFFECTS YOU
●
builderYou can enhance the reasoning capabilities of small models with minimal parameter overhead.
●
researcherYou can apply this iterative latent transition method to improve reasoning in dense pretrained models.