ExpertCrowd-SWE benchmark for multi-file coding agent evaluation
November 11, 2026
ExpertCrowd-SWE introduces 100 expert-authored software engineering tasks across 87 open-source projects and 11 languages. The benchmark uses a task-specific grader to evaluate change requests in unfamiliar codebases, with a median reference solution spanning 471 lines.
HOW THIS AFFECTS YOU
●
builderYou can use this to test how well your coding agent handles real-world, multi-file repositories rather than simple single-file tasks.
●
researcherThis provides a more rigorous way to measure if agents actually understand large, unfamiliar codebases.