A new evaluation framework for RAG systems uses asymmetric scoring and knowledge-gap canaries to penalize models that guess instead of abstaining. Testing on SimpleQA-Verified shows that current volume-based accuracy metrics reward ungrounded generation.
HOW THIS AFFECTS YOU
●
builderYou should implement abstention policies to improve the reliability of your RAG products.
●
researcherYou can use knowledge-gap canaries to better distinguish between retrieval failures and parametric memory hallucinations.