Voynich Manuscript Analysis Rejects Standard Tokenization and Substitution Hypotheses
August 19, 2026
Analysis of the Voynich manuscript shows glyph regularity (2.7 bits conditional entropy) is too high for one-to-one substitution and token sequences fail to predict subsequent tokens at rates seen in control languages. The data suggests information is encoded in recurrent multi-symbol units rather than discrete words or letters.
HOW THIS AFFECTS YOU
●
researcherYou should reconsider standard NLP tokenization assumptions when analyzing non-standard or highly structured symbolic datasets.