FairSumEval introduces evidence-aware metrics, Soft IoU-S and Soft IoU-NS, to distinguish between models that omit information and those that add ungrounded content. Testing on 200 MIMIC-III admissions revealed that 18.8% of physician reference facts had identified support in the clinical notes.
HOW THIS AFFECTS YOU
●
researcherThis framework prevents you from penalizing models for not reproducing information that was never in the input.
●
healthThis provides a more rigorous way to check if medical summaries are hallucinating new clinical plans.