Can Released LLM Vocabularies Support Token-Level Estimation of Hidden Corpora?

Quantile-Guided Density Estimation (QGDE) estimates the composition of hidden pretraining corpora by mapping token ID-to-ratio distributions from known datasets to target tokenizers. Testing on the SmolLM tokenizer yielded mean relative errors below 3.08% for category-level mixtures, enabling more accurate inference of training data distributions from released vocabularies.