Modular Expert Merging for Specialized Biomedical Retrieval

The Synthesize-Train-Merge (STM) framework improves biomedical retrieval by merging independently trained LoRA experts instead of using large-scale mixed-domain training. This method uses synthesized hard negatives to capture complementary domain strengths in models ranging from 0.6B to 7B parameters.