Research shows that surface noise like typos can turn neutral LLM judgments into biased ones by up to 120x. The study demonstrates that LLM judges are disproportionately sensitive to noise when evaluating social bias.
HOW THIS AFFECTS YOU
●
researcherThis highlights a major reliability gap in using LLMs as automated evaluators of social fairness.
●
policyYou should account for text noise when using LLMs to audit social bias or compliance.