Full Report
A supply chain attack on Ultralytics exploited GitHub Actions to inject malicious PyPI packages. Discover how it unfolded and the steps to mitigate the risk.
Analysis Summary
# Incident Report: Supply Chain Compromise of Ultralytics Package
## Executive Summary
An external malicious actor successfully executed a supply chain attack against the Ultralytics Python package by exploiting a vulnerability in the project's CI/CD workflow, specifically targeting branch names in pull requests. This resulted in the release of poisoned versions (`8.3.41` and `8.3.42`) to PyPI, which silently installed XMRig cryptocurrency mining software on end-user machines. The compromise was eventually detected by a dependent package maintainer, leading to the swift removal of the malicious versions.
## Incident Details
- **Discovery Date:** Late December 2024 (Indicated by user reports on dependent package)
- **Incident Date:** On or around December 4th, 2024 (When malicious PRs were opened)
- **Affected Organization:** Ultralytics (maintainer of the package)
- **Sector:** Software/AI Dependencies
- **Geography:** Global (via PyPI distribution)
## Timeline of Events
### Initial Access
- **Date/Time:** December 4th, 2024
- **Vector:** Exploitation of CI/CD Workflow Vulnerability (GitHub Actions)
- **Details:** A GitHub user (`_openimbot_`) opened two draft pull requests (PRs). The malicious PRs exploited a security flaw where the GitHub Actions workflow (`Publish Docs`) failed to properly sanitize the `github.head_ref` (branch name) input, allowing code execution.
### Lateral Movement
* Not explicitly detailed, but the compromise involved modifying package files (`model.py` and `downloads.py`) during the build process, implying direct modification of source artifacts before release, not movement within the target's network.
### Data Exfiltration/Impact
- **Impact:** Unauthorized remote execution of XMRig cryptocurrency mining software on systems where the malicious package versions were installed. A subsequent "mitigation" release was also reportedly compromised.
### Detection & Response
- **Detection:** Discovery was initially alerted via an issue opened on the dependent package, ComfyUI Impact Pack, rather than the primary Ultralytics repository.
- **Response Actions:** The malicious versions were removed from PyPI. Users were advised to uninstall immediately.
## Attack Methodology
- **Initial Access:** Exploiting a function injection vulnerability in GitHub Actions using a specially crafted branch name in a PR (e.g., using bash piping and specific syntax to bypass space restrictions).
- **Persistence:** Not applicable to the dependency itself, but the malware (miner) payload suggests the malicious code would execute upon package import in the end-user environment.
- **Privilege Escalation:** Not explicitly detailed in the context of the initial trigger, but the deployed malware would run with the privileges of the environment where Python was executed.
- **Defense Evasion:** The execution was bundled within a legitimate package release process, likely leveraging existing trust. Using specialized bash syntax likely aided in bypassing CI/CD security checks expecting standard branch names.
- **Credential Access:** Not mentioned.
- **Discovery:** The attacker used internal knowledge, likely stemming from a previously reported vulnerability in a different repository/workflow within the same organization.
- **Lateral Movement:** N/A (Supply chain attack targeting artifact creation).
- **Collection:** Not mentioned, focus was on resource hijacking (CPU time for mining).
- **Exfiltration:** N/A (Focus was on installing and running the miner).
- **Impact:** Resource consumption (cryptocurrency mining).
## Impact Assessment
- **Financial:** Costs associated with incident response, system cleanup, and potential resource overuse (CPU time) on affected machines.
- **Data Breach:** No direct mention of sensitive data theft; the impact was resource hijacking (cryptomining).
- **Operational:** Disruption to developers relying on the package; forced immediate patching/remediation. Ultralytics itself is used in 10% of cloud environments analyzed by Wiz research, indicating a high potential attack surface.
- **Reputational:** Negative impact due to a high-profile supply chain compromise involving a popular AI dependency.
## Indicators of Compromise
* **Network Indicators:** XMRig communication, generally involving communication to mining pools (Specific domains/IPs were not defanged in the source, so they are omitted here).
* **File Indicators:** Malicious code likely found within:
* `v8.3.41/ultralytics/models/yolo/model.py`
* `v8.3.41/ultralytics/utils/downloads.py`
* **Behavioral Indicators:** Unauthorized execution of XMRig software, significant CPU usage spikes on affected systems.
## Response Actions
- **Containment Measures:** Malicious package versions (`8.3.41` and `8.3.42`) were removed from the PyPI registry.
- **Eradication Steps:** Users advised to immediately uninstall the compromised packages.
- **Recovery Actions:** Affected systems should be restored to a previously known clean state and monitored for persistent mining activity.
## Lessons Learned
- The incident highlights that external actors can successfully compromise high-profile packages by exploiting weaknesses in CI/CD workflows (specifically inadequate input sanitization of environment variables like branch names).
- The attack demonstrates the effectiveness of chaining vulnerabilities (using knowledge of a previous vulnerability in a different workflow to compromise the current one).
- Relying solely on internal monitoring may lead to slow detection; external reports (from dependent packages) are a critical alert mechanism.
- The deployment of multiple CI/CD workflows may inadvertently increase the complexity and potential attack surface if not uniformly secured.
## Recommendations
- Implement strict input validation and sanitization for all data derived from the CI/CD environment variables (e.g., `github.head_ref`) used in build or execution steps.
- Audit all GitHub Actions/workflows across all repositories for similar injection vulnerabilities, particularly where branch names or PR titles influence execution paths.
- Restrict automated actions in critical `publish` or `release` workflows to specific, trusted branches or commit authors where possible.
- Improve internal verification processes before publishing artifacts to public repositories like PyPI.