Evaluating five LLM-based review paradigms on ICLR 2025 submissions, this study finds that direct absolute rating matches comparative ranking in human correlation. Recalibrated direct rating achieves higher agreement than Elo-based systems while using 24% fewer API calls.
HOW THIS AFFECTS YOU
●
researcherYou can use recalibrated direct rating for more efficient and accurate automated manuscript screening.