LMEnt suite for analyzing knowledge acquisition in language models

LMEnt provides a knowledge-rich pretraining corpus and an entity-based retrieval method that outperforms existing tools by 80.4%. It includes 12 pretrained models up to 1B parameters with 4K checkpoints to track how entity mentions translate to downstream knowledge.