Audit Reveals 15.3% Error Rate in Computer-Use Agent Benchmarks

An audit of 150 trajectories across five benchmarks found that 15.3% of FAIL verdicts were incorrect due to evaluator false negatives or broken tasks. The study identifies that planning and verification failures, rather than execution errors, are the primary drivers of genuine agent failure.