LLMs Diverge from Human Experts in Security Survey Responses
August 19, 2026
A methodological framework evaluates LLMs as surrogate participants for security expert surveys, comparing persona-based and aggregate generation against SOC professionals. While LLMs show internal consistency, they systematically diverge from human expert responses in security operational contexts.
HOW THIS AFFECTS YOU
●
researcherYou should avoid using LLMs as direct substitutes for human experts in specialized security workforce studies.
●
policyThis highlights the risks of relying on synthetic data to model human behavior in critical security infrastructure.