SWE-Bench Pro Verified mitigates reward hacking and task quality issues found in the original SWE-Bench Pro through anti-hacking safeguards and task refinement. This creates a more reliable evaluation standard for software engineering agents by removing leaked solutions and inconsistent tests.
HOW THIS AFFECTS YOU
●
builderThis helps you avoid overestimating the performance of your coding assistants during evaluation.
●
researcherUse this version to get an unbiased measurement of your agent's true coding capabilities.