New Probing Protocol for Detecting Benchmark Contamination

A new nuisance-controlled residual-stream probing method improves benchmark contamination detection by using a zero-sum contrast on depth profiles. This approach prevents false positive rates from scaling wildly with the size of the control set, unlike previous linear probing methods.