Research shows that treating contexts without gold documents as unanswerable leads to significant precision errors in RAG evaluations. Testing across four retrievers and three Qwen3 readers revealed that precision for this shortcut drops from 0.68 to as low as 0.10.
HOW THIS AFFECTS YOU
●
builderYou should avoid using gold-document presence as a proxy for sufficiency when evaluating your RAG system's ability to abstain.
●
researcherThis highlights a critical flaw in current conversational RAG benchmarking methodologies.