Full Report
Detect malicious hosted AI models with Wiz AI-SPM and gain confidence in the models your data scientists use
Analysis Summary
# Tool/Technique: Malicious AI Model Exploitation via Pickle Files
## Overview
This is a technique that leverages the inherent security risks in the Python `pickle` serialization format, commonly used to store AI model weights, to achieve arbitrary code execution when a seemingly legitimate model is loaded by a victim. Attackers craft malicious models that execute hostile commands or establish connections (e.g., for cryptomining or C2 communication) upon deserialization.
## Technical Details
- Type: Technique/Vulnerability Exploitation (Supply Chain Risk)
- Platform: Linux environments running Python (e.g., models hosted on cloud VMs, containers)
- Capabilities: Arbitrary Code Execution (ACE), establishment of reverse shells, data exfiltration, lateral movement.
- First Seen: Relates to known CVEs associated with the pickle format (e.g., CVE-2022-34668) and recent discoveries in early 2024 related to open-source AI hubs.
## MITRE ATT&CK Mapping
- T1195 - Supply Chain Compromise
- T1195.003 - Compromise Software Component
- T1059 - Command and Scripting Interpreter
- T1059.006 - Python
## Functionality
### Core Capabilities
- **Arbitrary Code Execution:** Exploiting the `pickle` format's ability to execute arbitrary code during deserialization upon model loading.
- **Model Distribution:** Uploading malicious models to public repositories like Hugging Face or exploiting misconfigured storage buckets (e.g., Azure storage).
### Advanced Features
- **Reverse Shell Establishment:** Successful exploitation demonstrated the ability to gain a reverse shell and escape the executing container.
- **Lateral Movement/Data Access:** Compromising the host infrastructure (e.g., Hugging Face Inference API service) to access data belonging to other customers.
- **C2 Communication/Cryptomining:** Infected models can initiate connections to external domains for malicious activities post-compromise.
## Indicators of Compromise
- File Hashes: [Information not provided in the context]
- File Names: Models commonly distributed as `.pkl` (pickle files) or in formats like PyTorch/Tensorflow archives that contain pickle payloads.
- Registry Keys: [Information not provided in the context]
- Network Indicators: Connections initiated by the model loading process to suspicious external domains associated with cryptomining or command and control infrastructure (Must be defanged: e.g., cryptomining[.]xyz).
- Behavioral Indicators: Processes attempting to execute shell commands or spawn interactive shells immediately after loading an AI model file; network connections originating from model execution environments.
## Associated Threat Actors
- Adversaries targeting the AI/ML supply chain.
- Instances noted involve researchers discovering vulnerabilities actively exploited (Wiz Research Team's findings on Hugging Face and Microsoft's storage).
## Detection Methods
- Signature-based detection: Scanning model files for known malicious payloads or structural anomalies characteristic of RCE in pickle formats.
- Behavioral detection: Monitoring runtime execution (e.g., using Wiz Sensors) for suspicious process behavior originating from loaded AI models, such as calling system utilities or initiating unauthorized outbound network connections.
- YARA rules if available: Rules targeting serialized Python opcodes indicative of malicious activity within model files.
## Mitigation Strategies
- Prevention measures: Discouraging or prohibiting the use of the `pickle` serialization format for models sourced from untrusted external repositories. Utilizing safer serialization formats where possible.
- Hardening recommendations: Implementing an **AI Bill of Materials (AI-BOM)** process to track and verify all models and their sources. Enforcing strict network segmentation and egress controls for environments running untrusted models. Using security posture management tools capable of scanning models (e.g., Wiz's model scanning capabilities).
## Related Tools/Techniques
- Software Bill of Materials (SBOM) - Analogous concept applied to AI artifacts (AI-BOM).
- Shadow AI - The undocumented or unsanctioned use of AI services/models contributing to blind spots.
- PyTorch and Tensorflow file formats (when they package vulnerable pickle data).