User Hostility Identified in 0.9% of LMSYS Chat Conversations

Auditing 777K LMSYS conversations reveals that lexicon-based detectors for model-directed insults and coercion capture 0.9% of user turns, distinct from standard toxicity moderation flags. The study shows that most moderation signals flag user-generated toxic content rather than direct hostility or jailbreak pressure aimed at the assistant.