A new label-free procedure increases inter-judge consensus by decomposing rubrics into sub-criteria and using an editor model to rewrite contested clauses. Testing across 129,770 requests on SummEval and HealthBench shows improved agreement among LLM panels without requiring human labels.
HOW THIS AFFECTS YOU
●
builderYou can use this decomposition method to stabilize evaluation pipelines when using LLM-as-a-judge.
●
researcherThis provides a way to repair evaluation rubrics without the cost of new human annotations.