[arXiv]score: 0.34

Synthetic Data Training Causes Catastrophic Forgetting in LLM Skill Routers

September 11, 2026

Fine-tuning LLM skill routers on synthetic data improves in-distribution retrieval but degrades out-of-distribution performance. Implementing EWC or L2-initialization can mitigate this, improving 0.6B Qwen retriever performance on synthetic skills by 13.98% while retaining OOD capabilities.

HOW THIS AFFECTS YOU

●

builderYou should use regularization techniques like EWC when training agents on synthetic skill datasets to avoid losing real-world utility.

●

researcherThis provides a practical benchmark for studying the trade-offs between synthetic data scaling and retrieval robustness.

read original ↗arxiv.org

DAILY DIGEST

you don't check 9 sources — we do. one email every morning, read in 2 min. free. unsubscribe anytime. privacy

← back to feed