Full Report
The author of this post is a member of the CTF team Shellphish. His team, a world-renowned one at that, had earned its way to compete in the DEFCON CTF this year. This is the Olympics of hacking and is home to many of the world's best CTF players. They had previously competed in the AIxCC competition, where LLMs attempted to identify bugs in code. With all of this in mind, they decided to tackle a pwn challenge called ico. This was a small binary but contained over 6K functions, making this a classic reversing challenge. Throughout the event, Blue Water had solved two of the Live CTF challenges (small one-on-one challenges) using agents running in the background. So, Wil decided to spin up some LLM infrastructure to see if it could be solved this way. They created a Docker container that contained the IDA MCP server and Cursor inside of it. They gave it a prompt of along the lines of "You are a great reverse engineer. Reverse the application and interact with the binary at this port when needed". After running GPT-5 for back and forth for a long time (with A LOT of tool calls), it outputted a script that did not work but had some good insights on the program. The author posts the exact prompts and output throughout the post, which is very nice to see. The LLM asked them to create a better script with pwntools for interacting with the challenge based on the information for the commands given by the LLM. This helped but there was still no flag. The LLM hadn't updated the decompilation at all. The author made several changes, including to function names, to provide the LLM with more context on how it works. After going back and forth a few more times with "we need the flag and not the MD5 hash of the flag", the LLM eventually figured out how to extract the flag from the challenge! Even cooler, they asked it to patch the binary and it was able to fix the challenge as well. Pretty neat! According to the author, this was a perfect storm: a straightforward path to exploitation with no tricks, just reversing, a simple exploit (just 10 bytes required), and the problem was partially reverse-engineered already. They claim this could be used to solve some CTF challenges, but not most of them. In general, the process of "gather knowledge (from IDA) -> formulate hypothesis -> create exploit script -> analyze script output -> apply new findings to IDA" worked pretty well for them.
Analysis Summary
# Tool/Technique: LLM-Assisted Automated Reverse Engineering (MCP-Integrated)
## Overview
This technique involves the orchestration of Large Language Models (LLMs), specifically GPT-5, integrated with binary analysis tools via the Model Context Protocol (MCP). By connecting an LLM to an IDA Pro MCP server within a unified environment (such as Cursor), the model can autonomously perform static analysis, formulate hypotheses about binary logic, and generate/execute functional exploit scripts to solve complex capture-the-flag (CTF) challenges.
## Technical Details
- **Type**: Technique / Attack Tool Framework
- **Platform**: Linux (x86-64)
- **Capabilities**: Automated decompilation analysis, autonomous tool calling, exploit script generation (pwntools), binary patching, and iterative hypothesis testing.
- **First Seen**: August 14, 2025 (DEF CON 33 CTF Finals)
## MITRE ATT&CK Mapping
- **[TA0007 - Discovery]**
- [T1083 - File and Directory Discovery]
- [T1057 - Process Discovery]
- **[TA0002 - Execution]**
- [T1059 - Command and Scripting Interpreter]
- **[TA0008 - Exploitation]**
- [T1203 - Exploitation for Client Execution]
- **[TA0003 - Persistence]**
- [T1542.001 - Pre-OS Boot: ROM披露 (Relevant via Binary Patching capability)]
## Functionality
### Core Capabilities
- **IDA Pro Integration**: Uses the Model Context Protocol (MCP) to allow the LLM to query function names, decompilation, and cross-references directly from a live IDA session.
- **Iterative Feedback Loop**: Adopts a cycle of "Gather knowledge -> Formulate hypothesis -> Create exploit script -> Analyze output -> Update IDA context."
- **Context Management**: The LLM can request manual updates to the IDB (e.g., renaming functions or types) to improve its own understanding of the code flow.
- **Automated Pwn**: Generates Python scripts using the `pwntools` library to interact with remote services and trigger vulnerabilities (e.g., buffer overflows).
### Advanced Features
- **Heuristic Bug Discovery**: Capable of identifying logic flaws or memory corruption vulnerabilities in binaries with high function counts (6K+) that lack modern protections like PIE or Stack Canaries.
- **Automated Patching**: Beyond exploitation, the LLM can calculate file offsets from Virtual Addresses (VAs) and generate scripts to modify binary opcodes to change program behavior (e.g., bypassing checks).
## Indicators of Compromise
- **File Hashes**:
- *Note: Targeted binary "ico" hash not provided, but methodology involves generating ad-hoc exploit scripts.*
- **File Names**: `poc_ncif.py`, `patch_simple.py`, `ico.bak`
- **Network Indicators**:
- Default challenge port: `4265`
- Local MCP Server communication: `127.0.0.1:[PORT]`
- **Behavioral Indicators**: High-frequency tool calls from LLM agents to IDA Pro; repeated automated connections to a target service with varying payload lengths to test for overflows.
## Associated Threat Actors
- **Research/CTF Groups**: Shellphish, Blue Water.
- **Note**: While currently used for security research, this framework mirrors the "AI-driven autonomous agent" capabilities sought by advanced persistent threats for rapid vulnerability research.
## Detection Methods
- **Behavioral Detection**: Monitoring for "machine-speed" interaction with debugging or disassembly tools. Detecting the deployment of IDE-integrated LLM agents (like Cursor) in sensitive development or production environments.
- **Network Traffic**: Identifying anomalous repetitive traffic patterns characteristic of automated fuzzing or LLM-generated exploit attempts against internal services.
## Mitigation Strategies
- **Binary Hardening**: Enable Position Independent Executables (PIE), Stack Canaries, and Full RELRO to increase the complexity required for LLM-generated exploits.
- **Anti-Symbolic/Anti-AI Techniques**: Implement obfuscation or complex state machines (VM-based instruction sets) that increase the "token cost" and reasoning complexity for LLMs.
- **Environment Isolation**: Prevent LLM agents from accessing local network interfaces or sensitive binary analysis tools without strict human-in-the-loop oversight.
## Related Tools/Techniques
- **IDA Pro MCP**: The bridge between the disassembler and the LLM.
- **Cursor**: The AI-integrated code editor used as the primary interface.
- **Angr**: A binary analysis framework often used in conjunction with autonomous hacking.
- **AIxCC (AI Cyber Challenge)**: The DARPA competition focused on LLM-driven vulnerability discovery.