System 1 Decision Model for Medical MCQs using Frozen LLMs
November 19, 2026
A single-pass model uses two trained projection heads on frozen Qwen3.5-0.8B, Granite-4.2-3B, and Qwen3-8B to solve medical questions. On MedMCQA, the small encoders achieved 33.1% and 33.0% accuracy, outperforming the 25.0% chance level.
HOW THIS AFFECTS YOU
●
builderYou can achieve slightly better than chance performance on medical exams using very small models like 0.8B parameters.
●
researcherYou can use frozen small encoders and contrastive loss to handle medical knowledge tasks without full fine-tuning.