Latency-Minimizing Request Scheduling for Edge LLM Inference

An online scheduling framework for edge LLM inference jointly minimizes end-to-end latency and balances workloads across heterogeneous distributed servers. It addresses the challenges of multi-stage LLM execution dynamics and the delayed observation of scheduling consequences in agentic AI services.