18 Frontier MLLMs Flip Answers 24–50% of the Time Under Input Reordering

Facet-Probe audits 18 MLLMs across five ordering perturbations — option, evidence-chunk, document-rank, image-set, and mixed-modality — finding flip rates of 24–50% panel-mean across all models. Even the best model flips on 13.4% of trials, and a Gemini temperature-0 control confirms the excess is ordering-driven, not decoder stochasticity. Standard benchmarks using single canonical orderings systematically miss this reliability failure.