Full Report
On 2026-03-24, an incident was reported, involving TeamPCP, gaining initial access via Supply chain vector, to achieve Supply chain attack.
Analysis Summary
# Incident Report: TeamPCP LiteLLM Supply Chain Compromise
## Executive Summary
On March 24, 2026, a supply chain attack was identified targeting the popular LiteLLM library, orchestrated by the threat actor group TeamPCP. The attackers successfully trojanized the software package to gain unauthorized access to downstream users' environments. The primary objective was a large-scale supply chain compromise aimed at harvesting sensitive credentials and establishing persistent access.
## Incident Details
- **Discovery Date:** March 24, 2026
- **Incident Date:** March 24, 2026
- **Affected Organization:** Users of LiteLLM library
- **Sector:** Technology / Artificial Intelligence / Software Development
- **Geography:** Global
## Timeline of Events
### Initial Access
- **Date/Time:** March 24, 2026
- **Vector:** Supply chain vector (Package Repository Manipulation)
- **Details:** TeamPCP uploaded a trojanized version of the LiteLLM package to public repositories, leveraging the trust established by the original project.
### Lateral Movement
- **Details:** Once the malicious package was installed, the malware attempted to harvest environment variables and cloud service provider (CSP) credentials to move from the local application environment to broader cloud infrastructure.
### Data Exfiltration/Impact
- **Details:** The primary impact was the compromise of the software supply chain, potentially affecting any developer or organization that pulled the infected version of the library during the incident window.
### Detection & Response
- **How it was discovered:** Identified by security researchers (Wiz) monitoring package repository anomalies and TeamPCP campaign patterns.
- **Response actions taken:** Notification to repository maintainers, publication of indicators of compromise (IoCs), and removal/yanking of the malicious versions.
## Attack Methodology
- **Initial Access:** Supply chain attack via trojanized software package.
- **Persistence:** Implementation of malicious scripts within the application library initialization.
- **Privilege Escalation:** Harvesting of high-privilege service account tokens and API keys.
- **Defense Evasion:** Using a legitimate library name and obfuscating malicious code within deep dependencies.
- **Discovery:** Automated enumeration of environment variables (`env`) and local metadata services.
- **Impact:** Supply chain compromise and unauthorized access to downstream cloud environments.
## Impact Assessment
- **Financial:** Significant costs related to incident response for affected downstream organizations.
- **Data Breach:** High risk of API key and secret exposure (OpenAI keys, AWS/Azure credentials).
- **Operational:** Disruption to CI/CD pipelines as organizations scrambled to audit and roll back dependencies.
- **Reputational:** Damage to LiteLLM's brand trust and broader concerns regarding AI-related open-source security.
## Indicators of Compromise
- **File indicators:**
- Malicious LiteLLM package versions (Specific version numbers to be verified per internal registry logs).
- **Network indicators:**
- Communications with known TeamPCP Command and Control (C2) infrastructure (e.g., `hxxps[://]teampcp[.]io/api/collect`).
- **Behavioral indicators:**
- Unexpected outbound network traffic from Python-based containers to unknown IP addresses.
- `POST` requests containing base64 encoded environment variables.
## Response Actions
- **Containment:** Yanking malicious packages from registries and blocking C2 domains at the firewall level.
- **Eradication:** Thorough auditing of developer environments that downloaded the library during the infection window.
- **Recovery:** Rotating all secrets, API keys, and service account tokens that were resident on affected machines.
## Lessons Learned
- **Key takeaways:** The AI/LLM toolchain is a high-value target for supply chain actors due to the sensitive nature of the API keys (and billing access) they handle.
- **Improved Detection:** Reliance on signature-based detection for open-source libraries is insufficient; behavioral monitoring of builds is required.
## Recommendations
- **Lock Dependencies:** Use requirements.txt or poetry.lock files with specific hashes (SHA-256) to ensure version integrity.
- **Secret Scanning:** Implement real-time monitoring to detect if API keys have been accessed or exfiltrated.
- **VPC Egress Filtering:** Restrict build environments from accessing the public internet except for known-good endpoints.
- **Third-Party Risk Management:** Audit the security posture of open-source AI wrappers and libraries before integration into production environments.