Serving-Context Nondeterminism in Text Classifiers
October 8, 2026
A systematic study of 180 models reveals that serving contexts like batch size, hardware, and inference engine implementation cause non-deterministic outputs in text classifiers. These inconsistencies stem from floating-point non-associativity and shape-dependent kernel selection during numerical execution.
HOW THIS AFFECTS YOU
●
builderYou can no longer assume identical classification results across different batch sizes or hardware setups.
●
researcherYou should account for implementation-level numerical variances when evaluating classifier stability.