GAUSS: Workload-Aware Simulator for LLM Inference Latency and Throughput
October 30, 2026
GAUSS is an analytical and Monte-Carlo simulator that predicts full distributions for TTFT and TBT by modeling prefill and decode phases separately. It accounts for continuous batching and stochastic arrivals to estimate sustainable request rates under specific SLOs.
HOW THIS AFFECTS YOU
●
builderYou can use this to model and predict tail latency and throughput for your production inference fleets.
●
founderThis helps you more accurately estimate infrastructure costs and service level guarantees for customers.