Performance Gap in System One Models for Indonesian Regional Languages
October 14, 2026
Evaluation of Jev and Laya-multilingual models shows accuracy improves by up to 37.2 points when providing human Indonesian translations for regional languages. While expected calibration error remains stable, the models' ability to rank their own errors degrades significantly on non-English regional datasets.
HOW THIS AFFECTS YOU
●
builderTranslating inputs to a dominant language can significantly boost performance for low-cost classifier models.
●
researcherYou should account for error-ranking degradation when evaluating low-resource language performance.