Medical LLM Rubrics Fail to Detect Clinically Relevant Hallucinations

Analysis of MedHallu and HealthBench shows that standard rubric-based evaluations often miss clinically significant hallucinations. The study demonstrates that while rubrics capture factual errors, they fail to detect unexpected or additional errors, necessitating more specialized, clinician-validated evaluation pipelines.