Evaluating Language Models in Realistic Conversational Contexts

UPHELD provides a benchmark of 30,000 expert-generated dialogue turns and 36,000 per-turn human annotations to measure multi-turn conversational consistency. Unlike synthetic datasets, this benchmark utilizes complete human-to-human dialogues authored by professional scriptwriters to evaluate model performance beyond simple factual correctness.