Formalizing LLM Creativity via Long-Tail Performance Metrics

Creativity is quantified as task-specific performance in the long tail of an input-output distribution. Using n-gram absence from pretraining data as a novelty proxy, an evaluation of 26 open-weight models across 15 benchmarks shows creativity tasks reside further in the tail than standard benchmarks.