Full Report
Programs leveraging AI agents are increasingly popular. Nine attack scenarios using open-source agent frameworks show how bad actors target these applications. The post AI Agents Are Here. So Are the Threats. appeared first on Unit 42.
Analysis Summary
# Tool/Technique: Agentic Applications & AI Agents
## Overview
This summary focuses on the security implications and attack vectors targeting **Agentic Applications**, which are defined as programs leveraging **AI Agents**—software designed to autonomously collect data and take actions toward specific objectives. The research implements attacks against functionally identical applications built using the open-source agent frameworks **CrewAI** and **AutoGen** to demonstrate framework-agnostic vulnerabilities arising from design patterns, misconfigurations, and unsafe tool integrations.
## Technical Details
- Type: Technique (Attacks against Agentic Applications)
- Platform: Framework-agnostic, targeting applications built on AI agent frameworks (e.g., CrewAI, AutoGen).
- Capabilities: Attackers aim for information leakage, credential theft, tool exploitation, and Remote Code Execution (RCE) via insecure design patterns in agent implementations.
- First Seen: The concepts surrounding AI agent security are emerging with the widespread adoption of Generative AI and AI Agents (context date: May 1, 2025).
## MITRE ATT&CK Mapping
The techniques described primarily map to the Initial Access, Execution, Discovery, and Impact tactics related to compromising the underlying system or data via the agent interface.
- **TA0001 - Initial Access**
- **T1560 - Archive Collected Data** (Related to data exfiltration preparation)
- **TA0002 - Execution**
- **T1059 - Command and Scripting Interpreter** (Related to arbitrary code execution via insecure code interpreters)
- **TA0009 - Collection**
- **T1005 - Data from Local System** (Related to information or credential leakage)
- **TA0011 - Command and Control**
- (Implied C2 activity resulting from compromised tooling or RCE)
- **TA0006 - Credential Access**
- **T1552 - Unsecured Credentials** (Direct target for credential theft)
## Functionality
### Core Capabilities (Attack Vectors)
- **Prompt Injection:** Remains a potent vector to leak data, misuse tools, or subvert agent behavior.
- **Misconfigured/Vulnerable Tools:** Exploiting third-party tools integrated by the agent, increasing the attack surface.
- **Unsecured Code Interpreters:** Exploiting interpreters to achieve arbitrary code execution and access host resources.
- **Credential Leakage:** Exposing service tokens, secrets, or sensitive information within the agent's operational scope.
- **Poorly Scoped/Unsecured Prompts:** Exploitation without explicit injection, leveraging broad or weak instruction boundaries.
### Advanced Features (Outcomes of Successful Attacks)
- Information leakage.
- Credential theft and subsequent impersonation/privilege escalation.
- Tool exploitation.
- Remote Code Execution (RCE).
- Unauthorized access to host networks and resources.
## Indicators of Compromise
The provided context focuses on **vulnerabilities and attack patterns** rather than specific malware samples, so hard IOCs are generally framework-agnostic.
- File Hashes: N/A (Focus on application logic and design flaws)
- File Names: N/A
- Registry Keys: N/A
- Network Indicators: Attack payload destinations related to successful RCE or data exfiltration (specific domains/IPs not provided in the summary).
- Behavioral Indicators:
- Execution of shell commands or scripts via agent tools when not expected.
- Unexpected network connections initiated by the agent process.
- Attempts to access or read configuration/secret files outside normal operational boundaries.
- Output containing system instructions or tool schemas.
## Associated Threat Actors
Not explicitly named, but these risks apply to any threat actor targeting:
- Applications built using **CrewAI** or **AutoGen** frameworks.
- Organizations deploying **Agentic AI Applications**.
## Detection Methods
Detection relies heavily on runtime monitoring and secure coding practices:
- **Signature-based detection:** Limited application as attacks exploit logic flaws.
- **Behavioral detection:** Monitoring agent execution behavior, tool usage, and file system/network access for anomalous activity.
- **Content Filtering:** Deploying filters at runtime to detect and block known prompt injection payloads.
- **SAST/DAST/SCA:** Used proactively to find misconfigurations and vulnerable external tools integrated by the agent.
## Mitigation Strategies
A layered, defense-in-depth strategy is mandatory:
1. **Instruction Safeguards:** Enforce explicit instructions blocking out-of-scope requests and extraction of agent instructions/tool schema.
2. **Prompt Filtering:** Deploy runtime content filters for prompt injection attempts.
3. **Tool Security:** Sanitize all tool inputs, apply strict access controls, and secure tool integrations (via SAST/DAST/SCA).
4. **Code Interpreter Hardening:** Enforce strong sandboxing, network restrictions, syscall filtering, and least-privilege containerization.
5. **Credential Protection:** Implement DLP, audit logs, and use secret management services for all service tokens and secrets.
## Related Tools/Techniques
- **CrewAI:** Specific open-source agent framework tested.
- **AutoGen:** Specific open-source agent framework tested.
- **Prompt Injection:** Fundamental attack technique leveraged.
- **GenAI/BOLA (Broken Object Level Authorization):** Related security concepts in modern AI applications.