TagInsert enables dynamic generation order for sequence labeling
October 16, 2026
TagInsert removes the left-to-right decoder attention mask to allow Transformers to determine an optimal tag generation order during inference. In low-resource Part-of-Speech and CCG supertagging tasks, it matches encoder-only performance and improves results on long-tail rare categories.
HOW THIS AFFECTS YOU
●
researcherYou can move beyond fixed left-to-right generation constraints to improve performance in specialized sequence labeling tasks.