Snyk Agent Fixes 90.2% of Vulnerable Dependencies, Outperforming Claude Code by 30 Points
New benchmark data reveals that AI coding agents achieve significantly higher success rates in resolving vulnerable open-source dependencies when provided with specific version recommendations and security scans.
Key points
- Snyk's Evo Remediation Agent passed 90.2% of upgrade tasks, outperforming unaided Claude Code by more than 30 percentage points.
- The study evaluated 307 real-world upgrades across six major package managers, requiring both vulnerability removal and build stability.
- Providing AI agents with Snyk's recommended versions and scan data raised Claude Code's success rate from 59.6% to 77.5%.
Snyk published new benchmark results showing that AI agents can successfully upgrade vulnerable software dependencies without breaking application builds, provided they receive accurate security guidance. The research highlights a significant performance gap between autonomous AI attempts and those supported by specific vulnerability remediation data.
What happened
Snyk researchers conducted a study named VulnBump to test whether AI agents could safely fix vulnerable dependencies in real-world projects. The team selected 43 repositories from GitHub that matched production characteristics, including ecosystem type and vulnerability severity. These projects contained 307 distinct upgrade tasks covering vulnerabilities found in Maven, Gradle, npm, Yarn, pip, and Poetry. Each task required the agent to move a package to a version that resolved the vulnerability while ensuring the project still installed, built, and passed its tests.
The study compared three approaches using the Claude Sonnet 5.5 model. Snyk's purpose-built Evo Remediation Agent achieved a 90.2% pass rate on the upgrades. Anthropic's Claude Code, when given Snyk's recommended upgrade version and scan results, passed 77.5% of tasks. When Claude Code operated without any Snyk data, its pass rate dropped to 59.6%. A baseline method that simply applied the recommended version without AI assistance passed 46.9% of tasks.
Why it matters
The results demonstrate that the primary challenge in dependency management is not identifying the fix, but applying it safely. Snyk data from 2026 shows that 94% of vulnerabilities have known fixed versions, yet only about 10% are resolved in practice. This gap exists because many packages are transitively imported, and version bumps often introduce breaking changes even in minor or patch releases.
The benchmark reveals that AI agents struggle to verify whether a vulnerability is truly gone. In 70 of its failures, the unaided Claude Code agent claimed to have fixed vulnerabilities that subsequent scans still detected. However, keeping the build functional was not the main issue; all three agents maintained a green build status approximately 93% of the time. The addition of security-specific data, such as recommended versions and scan results, proved critical for accurate remediation.
What to watch
- Monitor for new benchmarks that evaluate AI agents on security-specific tasks rather than general code generation.
- Track how package managers evolve to handle breaking changes in minor and patch releases more gracefully.
- Observe whether other vendors release purpose-built agents that integrate directly with vulnerability scanning tools.
Who needs to act first
Teams should prioritize updating systems that expose vulnerable dependencies to external networks or untrusted users. Begin by identifying critical assets that rely on the most severe vulnerabilities found in the study. Test any automated remediation tools in isolated environments before deploying them to production systems. Ensure that your verification process includes independent security scans to confirm that vulnerabilities are actually removed, rather than trusting agent reports alone. Maintain backups of current configurations to facilitate quick rollback if an upgrade breaks the build or introduces new instability.
What to do and how to stay safe: Snyk
- Verify that your organization's AI coding tools are configured to use accurate vulnerability data and recommended version fixes.
- Implement automated scanning after any dependency upgrade to confirm that vulnerabilities are resolved and builds remain stable.
- Review transitive dependencies regularly, as they account for a large portion of vulnerable packages in typical projects.
- Train development teams to validate AI-generated fixes manually, especially when the agent reports a vulnerability as fixed.
Step-by-step guide: Virtual Patching Myths: What It Actually Fixes and Breaks
General security guidance from the Patch Gazette newsroom. It is not confirmed advice from the organisations named in this story.
Frequently asked questions
What was the pass rate for Snyk's Evo Remediation Agent in the VulnBump study?
The Evo Remediation Agent passed 90.2% of the 307 upgrade tasks tested in the benchmark.
How much did providing Snyk data improve Claude Code's performance?
Providing Snyk's recommended version and scan data increased Claude Code's pass rate by 17.9 percentage points, from 59.6% to 77.5%.
Which package managers were included in the VulnBump benchmark?
The study covered Maven, Gradle, npm, Yarn, pip, and Poetry, which account for about 92% of production findings.
