Evaluating LLM Orchestration Gains Against Cost and Optimization Effort

A controlled study of Self-Refine, Best-of-N, and Debate shows that orchestration yields moderate gains, averaging 4.5 to 4.6 percentage points over optimized Chain-of-Thought across programming, chess, and math. These improvements must be weighed against the significantly higher inference-time computation costs required.