Systematic Bias in LLM Advertising Relevance Judgments
October 7, 2026
Testing GPT-4o and Qwen-7B reveals that changing advertiser identity, language, or demographic wording alters relevance assessments. The study finds that LLM-based advertising judges exhibit biases consistent with common stereotypes in sensitive sectors like housing, employment, and credit.
HOW THIS AFFECTS YOU
●
builderBe aware that using LLMs as automated ad judges can introduce legal and fairness risks.
●
policyThis highlights the need for auditing LLMs used in high-stakes commercial decision-making.