OpenMTB-Audit Reveals LLM Over-Refusal in Oncology Safety Evaluations

An evaluation of 500 synthetic lung cancer cases shows LLMs fail to distinguish partially supported clinical recommendations from unsupported ones, with 83.3-100% of models collapsing labels to avoid risk. This reveals that high safety scores in medical LLMs often stem from excessive refusal rather than nuanced clinical reasoning.