Goldsmith Automates Annotation Definition Optimization via Textual Gradients
October 8, 2026
Goldsmith is an agentic pipeline that optimizes task definitions by using an LLM editor to convert structured loss failures into textual-gradient revisions. The system outperforms OPRO and PromptBreeder by treating the annotation definition as a trainable object optimized against a gold set.
HOW THIS AFFECTS YOU
●
builderYou can automate the creation of high-quality annotation guidelines using a small set of expert examples.
●
researcherYou can leverage textual-gradient revisions to iteratively refine task boundaries for supervised learning datasets.