Prior Scores Cause Anchoring Bias in LLM-as-a-Judge Evaluations

Evaluation-as-a-judge systems suffer from systematic bias where prior scores anchor subsequent ratings, even when presented only as metadata. Analysis of 185,271 evaluations shows a Cohen's d of 0.71, indicating significant shifts in score distributions.