Local Hidden-State Probe Outperforms GPT-3.5 for Metadata RAG Filtering
July 7, 2026
A 135M-parameter probe trained on small LLM hidden states achieves 90.9% accuracy in metadata filtering for multi-hop RAG. It outperforms GPT-3.5 by providing deterministic, fixed-vocabulary outputs and eliminating null-query errors common in prompted models.
HOW THIS AFFECTS YOU
●
builderYou can replace expensive, non-deterministic LLM calls with a lightweight, local probe for RAG metadata extraction.