BiPA Analysis Reveals Skill Optimizers Fail to Improve Agentic Tasks

The BiPA framework demonstrates that current skill optimizers, which rewrite natural-language procedures to improve success, show negligible real-world gains. Testing across three benchmarks showed learned skills succeeded only 0.1 points higher on average than hand-written seeds, with reported dev gains vanishing on test sets.