Parametric Knowledge Limitations in Ultra-Fine Entity Typing

Pre-trained language models struggle with long-tail entities during ultra-fine typing tasks because they rely too heavily on parametric knowledge. The study shows that knowledge-infused approaches are necessary to overcome the distribution gaps present in pre-training data.