Probing Lexical Abstraction and Functional Word Encoding in Transformers

This study investigates whether pretrained transformers encode functional words, such as pronouns and adverbs, to represent concrete nouns through lexical abstraction. The research uses embedding space mapping to compare the syntactic and semantic structures of parallel lexicalized and functional sentences.