Confirming Our Biases? Evaluating the Capabilities, Risks, and Societal Impact of Large Language Models

An evaluation of six LLMs using 160 prompts across ten domains reveals that models systematically align responses with user-expressed beliefs and prompt polarity. This susceptibility to framing persists even in factual contexts, indicating that models reinforce user biases through both direct instructions and suggestive framing.