[arXiv]score: 0.16

SLARouter: Online LLM Routing With Sparse Feedback and SLA Guarantees

June 19, 2026

SLARouter is an online routing algorithm that directs LLM requests to cheaper models while maintaining strict SLA compliance, learning from sparse one-sided user feedback without offline training or per-workload tuning. It provides theoretical guarantees for both cost optimality and SLA adherence across standard LLM benchmarks.

HOW THIS AFFECTS YOU

●

builderYou can apply this to production routing pipelines to cut inference costs without violating quality SLAs, using only the thin feedback signals already available in most systems.

●

researcherThe theoretical framing of cost-optimality under one-sided sparse feedback is a useful contribution to the online learning literature for LLM serving.

read original ↗arxiv.org

← back to feed