Improving Checklist Quality via Human-LLM Disagreement Patterns

This study utilizes patterns of disagreement between LLMs and experts to identify and refine ambiguous items in the GRoLTS appraisal checklist. The method demonstrates that analyzing where LLMs deviate from human judgment can systematically improve the clarity of quality appraisal frameworks.