Evaluating LLMs as Machine Translation Quality Annotators
October 11, 2026
An evaluation of LLMs using Multidimensional Quality Metrics (MQM) and Error Span Annotation (ESA) finds that while LLMs are cost-effective, they still face significant challenges matching human judgment in machine translation evaluation.
HOW THIS AFFECTS YOU
●
researcherAvoid relying solely on LLM-based MT metrics without rigorous validation against human MQM standards.