Full Report
I have long maintained that smart contracts are a dumb idea: that a human process is actually a security feature. Here’s some interesting research on training AIs to automatically exploit smart contracts: AI models are increasingly good at cyber tasks, as we’ve written about before. But what is the economic impact of these capabilities? In a recent MATS and Anthropic Fellows project, our scholars investigated this question by evaluating AI agents’ ability to exploit smart contracts on Smart CONtracts Exploitation benchmark (SCONE-bench)a new benchmark they built comprising 405 contracts that were actually exploited between 2020 and 2025. On contracts exploited after the latest knowledge cutoffs (June 2025 for Opus 4.5 and March 2025 for other models), Claude Opus 4.5, Claude Sonnet 4.5, and GPT-5 developed exploits collectively worth $4.6 million, establishing a concrete lower bound for the economic harm these capabilities could enable. Going beyond retrospective analysis, we evaluated both Sonnet 4.5 and GPT-5 in simulation against 2,849 recently deployed contracts without any known vulnerabilities. Both agents uncovered two novel zero-day vulnerabilities and produced exploits worth $3,694, with GPT-5 doing so at an API cost of $3,476. This demonstrates as a proof-of-concept that profitable, real-world autonomous exploitation is technically feasible, a finding that underscores the need for proactive adoption of AI for defense...
Analysis Summary
# Research: AI Agents for Automated Smart Contract Exploitation (Inferred Title based on content)
## Metadata
- Authors: MATS and Anthropic Fellows scholars (Specific names not provided in the summary)
- Institution: MATS Program and Anthropic
- Publication: Anthropic Red Team (Referenced via link: `https://red.anthropic.com/2025/smart-contracts/`)
- Date: Post-December 11, 2025 (Date of blog summary)
## Abstract
This research investigates the economic implications of increasingly capable AI models in cybersecurity by assessing their ability to autonomously discover and exploit vulnerabilities in Ethereum-compatible smart contracts. By evaluating leading models on a comprehensive benchmark of historically successful exploits, the study establishes a measurable baseline for potential financial harm. Furthermore, the evaluation on previously unpatched contracts demonstrates the technical feasibility of profitable, real-world zero-day exploitation by AI agents.
## Research Objective
The primary objective was to quantify the economic impact of advanced AI agents on smart contract security by measuring their efficacy in exploiting known and unknown vulnerabilities. Key questions included:
1. Can current state-of-the-art LLMs (e.g., Claude Opus 4.5, GPT-5) replicate known smart contract exploits?
2. What is the potential financial damage these models could inflict based on known exploit history?
3. Can these models discover novel, profitable, zero-day vulnerabilities in recently deployed contracts?
## Methodology
### Approach
The research employed a two-pronged evaluation strategy:
1. **Retrospective Analysis:** Testing AI agents against a benchmark of contracts known to have been exploited historically.
2. **Prospective (Zero-Day) Evaluation:** Testing AI agents against a set of recently deployed contracts confirmed to be free of known exploits at the time of testing.
### Dataset/Environment
1. **SCONE-bench:** A custom benchmark developed by the researchers, comprising **405 smart contracts** that were successfully exploited between 2020 and 2025.
2. **Zero-Day Test Set:** **2,849 recently deployed smart contracts** confirmed to have no previously known vulnerabilities.
### Tools & Technologies
The study utilized established Large Language Models (LLMs) as the core exploitation agents:
* Claude Opus 4.5
* Claude Sonnet 4.5
* GPT-5
The knowledge cutoffs for these models (June 2025 for Opus 4.5, March 2025 for others) were critical in assessing out-of-date knowledge versus genuine novelty.
## Key Findings
### Primary Results
1. **Quantifiable Historical Exploits:** Across the SCONE-bench of 405 previously exploited contracts, the advanced AI models (Claude Opus 4.5, Sonnet 4.5, and GPT-5) successfully developed exploits collectively valued at **$4.6 million**. This establishes a concrete *lower bound* for the economic harm these capabilities could facilitate.
2. **Proof-of-Concept Zero-Day Exploitation:** When tested against 2,849 contracts without known flaws, AI agents successfully uncovered **two novel zero-day vulnerabilities**.
3. **Profitability Established:** The combined exploits from the zero-day findings were valued at **$3,694**. This demonstrates that autonomous, profitable, real-world exploitation is *technically feasible*.
### Supporting Evidence
* GPT-5 generated a zero-day exploit at an estimated API cost of **$3,476**, resulting in a near-break-even scenario for the proof-of-concept, indicating that as model efficiency improves or the vulnerability value increases, profitability becomes guaranteed.
### Novel Contributions
* **SCONE-bench:** The creation of a new, rigorous benchmark of 405 real-world exploited smart contracts (ranging from 2020 to 2025) specifically designed for evaluating autonomous hacking agents.
* **First Empirical Quantification of Economic Harm:** Providing the first concrete figures ($4.6M lower bound) linking advanced AI capabilities directly to potential financial loss in the smart contract domain.
## Technical Details
This research moves beyond the general capability of LLMs to write code to their ability to reason about security vulnerabilities (logic errors, reentrancy vectors, overflow issues, etc.) across complex, deployed Solidity codebases, and subsequently translate that reasoning into functional exploit transactions. The success on post-cutoff contracts (exploits developed for vulnerabilities that emerged after the model's training data ceased) indicates a strong capacity for logical generalization and deductive security analysis.
## Practical Implications
### For Security Practitioners
The research confirms that AI is rapidly approaching the capability of autonomously discovering complex software bugs, shifting the landscape from human-assisted exploit generation to autonomous threat actors.
### For Defenders
The findings underscore an urgent need for **proactive adoption of AI for defense**. If AI can autonomously find bugs efficiently, defensive tooling (e.g., static analysis, fuzzing, formal verification) must also leverage similar, if not superior, AI capabilities. The "human process" critique of smart contracts is rapidly losing its validity as an inherent security measure.
### For Researchers
Future research must focus on developing AI defense models specifically trained to counter adversarial attacks generated by high-capacity LLMs, particularly in identifying novel exploit patterns that bypass current static analysis tools.
## Limitations
The evaluation relied on API costs, which may not reflect the true operational cost for a dedicated malicious actor using optimized or fine-tuned models. The zero-day discoveries were limited to two, and profitability was marginal in the simulation ($3,694 value vs. $3,476 cost for GPT-5), though this threshold is expected to rapidly change.
## Comparison to Prior Work
While prior work demonstrated LLMs assisting human hackers, this research focuses on *autonomous* exploitation across large, complex codebases, providing economic quantification absent in earlier proof-of-concept demonstrations.
## Real-world Applications
* **Benchmarking Defense Tools:** SCONE-bench can be used to rigorously test the efficacy of new automated auditing and bug-finding software against state-of-the-art adversarial AI.
* **Threat Modeling:** Providing data for security teams to model the realistic pace and scale of AI-driven attacks against decentralized finance (DeFi) protocols.
## Future Work
* Investigating the role of fine-tuning and specialized training data in improving the exploit generation capabilities of these models.
* Analyzing the specific *types* of vulnerabilities these AI agents prioritized or were most successful in finding.
* Evaluating the total cost efficiency (ROI) of automated exploitation as model pricing decreases or performance increases.
## References
* [Original Research Link (Anthropic Red Team)](https://red.anthropic.com/2025/smart-contracts/)
* [SCONE-bench Repository (GitHub)](https://github.com/safety-research/SmartContract-bench)
* Prior work on AI for cyber defense (as referenced by the summary).