LLM Judges Overestimate Text Summarization Faithfulness

Research shows human faithfulness annotations for summarization do not follow a strict conjunctive rule where every sentence must be supported. Instead, human labels correlate more closely with the average of per-sentence LLM judgments, suggesting humans accept summaries where most, but not all, sentences are faithful.