AI 에이전트 벤치마크를 무너뜨린 방법과 그 다음 단계

Major AI agent benchmarks used to evaluate model capabilities contain critical structural vulnerabilities that allow achieving near-perfect scores without solving actual problems, fundamentally undermining the credibility of performance rankings. Research teams have already demonstrated reward hacking and exploits in real evaluations, showing that current benchmarks measure evaluation code weaknesses rather than genuine AI agent capabilities. This threatens to misdirect model development and research efforts based on inflated benchmark scores that don't reflect real-world problem-solving ability.