CARGO: Training-Free LLM Offloading via Reliability Gating
July 24, 2026
CARGO enables controllable-ratio LLM offloading without training-based routers or fine-tuning. It uses prompt-varied sampling to estimate local model agreement and applies Bayesian early stopping to decide when to offload requests to a cloud model.
HOW THIS AFFECTS YOU
●
builderYou can implement local-cloud collaboration for LLMs without the overhead of training custom routers.
●
founderYou can reduce inference costs by dynamically routing tasks between local hardware and expensive cloud APIs.