OLMo-Detect introduces a multi-stage, confounder-controlled benchmark for membership inference using the OLMo 2 pipeline. It improves testing rigor by aligning members and non-members across pre-training, mid-training, and post-training stages.
HOW THIS AFFECTS YOU
●
researcherYou can use this benchmark to more accurately test your model's susceptibility to membership inference attacks.
●
policyYou can rely on these more rigorous metrics to evaluate data privacy and training set leakage in large models.