BPE Tokenizer Membership in Training Datasets via Rare Token Frequency
October 30, 2026
A threshold model of BPE inclusion shows that membership signals in a candidate dataset are primarily found in rare tokens that exceed merge thresholds. The RTAM-Fusion approach demonstrates that merge rank and graph position provide no first-order membership signal compared to token existence and frequency.
HOW THIS AFFECTS YOU
●
researcherThis provides a more precise statistical framework for analyzing data leakage through vocabularies.
●
policyThis method could be used to audit whether copyrighted or private data was included in a model's training set.