Defining the Recipe-Search Limit in Model Fine-Tuning

An analysis of 761 fine-tuning runs on models from 0.5B to 14B parameters identifies a mathematical limit where searching for optimal training recipes fails to improve performance. The research demonstrates that once recipe diameter falls below the resolution of scoring metrics, additional search yields no significant gains.