AssameseLM-E2B: 2B Model with 500% Tokenization Efficiency Gain
November 5, 2026
AssameseLM-E2B utilizes a Gemma-4 architecture with an expanded byte-level BPE vocabulary to reduce tokenization overhead by nearly 500% for the Assamese language. The 2B model achieves a 0.7488 BPB score on FLORES-200 and a 96.7% RAG score.
HOW THIS AFFECTS YOU
●
builderYou can deploy highly efficient, low-resource language support on consumer-grade hardware.
●
researcherThe use of Hybrid LoRA and expanded BPE vocabularies offers a template for low-resource adaptation.