LAMB introduces a Latin language encoder utilizing the ModernBERT architecture, pretrained on 74B Latin tokens. The release includes Acervus, a 24B token corpus with ByT5-repaired OCR transcriptions, and Corpus Trinum, the first Latin semantic triplet dataset.
HOW THIS AFFECTS YOU
●
researcherYou can use these specialized corpora and the morphologically informed masking regimen to advance low-resource NLP research.