Reference-Free LLM Framework for Benchmarking Conversational Agent Evaluators

This framework employs LLM judges to assess benchmark quality regarding consistency, complexity, and policy coverage. It uses actionable diagnostics to identify weaknesses in both manually curated and LLM-generated benchmarks, validating results against human annotations.