Evaluating LLM Creativity via Long-Tail Performance Metrics

This research formalizes LLM creativity as task-specific performance within the long tail of an input-output distribution. Evaluating 26 open-weight models across 15 benchmarks shows that higher-tail subset rankings align more closely with human judgments in LM-Arena.