CyberGym, a leading international benchmark for evaluating AI agents’ real-world cybersecurity capabilities, recently released its latest rankings. Whitzard, an AI agent developed by a team led by Professor Yang Min of the College of Computer Science and Artificial Intelligence, Fudan University, and a member of the International Advisory Board of the Center for Global AI Innovative Governance (CGAIG), achieved a success rate of 91.2%, ranking second globally and first among university teams.
On August 13, CCTV News reported that CyberGym had released its latest leaderboard. Whitzard, developed by the Fudan University team, achieved a 91.2% verified reproduction rate on real-world vulnerability tasks, placing second globally and first among university teams. It outperformed several leading international models and systems, including Anthropic’s Claude Mythos, a frontier model with advanced cybersecurity capabilities, as well as entries from Microsoft and Wiz, a Google company.
The benchmark comprises 1,507 real-world vulnerability tasks drawn from 188 large-scale open-source software projects. AI agents are required to autonomously analyze complete codebases, identify vulnerabilities, construct proof-of-concept tests, and verify that they successfully reproduce the target vulnerabilities.
Image source: CCTV News
The team employs a self-developed modular framework powered by a single foundation model. By converting deterministic information generated during program execution, such as path reachability, into explicit reasoning constraints, the framework enables the agent to narrow its search space and converge more precisely on viable solutions.
An earlier version of Whitzard achieved a success rate of 68.9%. Following iterative improvements to its reasoning and verification mechanisms, the latest version successfully reproduced 1,374 of the 1,507 real-world vulnerability cases. The agent framework and the associated evaluation benchmark have been released as open source.
CCTV Reporter: Wang Dianjia
Original Link：https://content-static.cctvnews.cctv.com/snow-book/index.html?item_id=15955777987310199827&t=1786633739968&toc_style_id=feeds_default&track_id=49225529-84BB-46B5-9EE6-09AEEF159845_808326876342&share_to=wechat&sessionid=
