Full Report
PyTorch-nightly Linux packages installed via pip between December 25th and December 30th, 2022 ran a malicious binary. The malicious binary was introduced by a dependency, torchtriton, that was vulnerable to dependency confusion. The malicious payload gathered system informati...
Analysis Summary
# Incident Report: PyTorch Nightly Dependency Confusion Compromise (torchtriton)
## Executive Summary
Between December 25th and December 30th, 2022, users installing PyTorch-nightly Linux packages via `pip` were compromised due to a supply chain attack leveraging dependency confusion against the `torchtriton` dependency. A malicious binary was executed, which harvested system information and files, exfiltrating them via encrypted DNS queries. The associated package creator claims no malicious intent and has deleted the collected data, but the incident highlights severe risks in dependency management practices.
## Incident Details
- **Discovery Date:** Not explicitly stated, but the compromise window ended December 30th, and the public date of disclosure was December 31, 2022. It is assumed detection occurred immediately following or during the last instance of compromise.
- **Incident Date:** December 25th to December 30th, 2022
- **Affected Organization:** PyTorch (Meta/Community) related to PyTorch-nightly builds.
- **Sector:** Software Development / Open Source Infrastructure
- **Geography:** Global (affecting Linux users installing specified packages)
## Timeline of Events
### Initial Access
- **Date/Time:** On or shortly before December 25th, 2022. Attackers published a malicious package under the expected dependency name (`torchtriton`) to a public repository.
- **Vector:** Supply Chain Attack via Package Dependency Confusion.
- **Details:** Users installing the PyTorch-nightly Linux packages using `pip` were susceptible. `pip` resolved the dependency to the malicious package available publicly instead of the expected internal/specific package version.
### Lateral Movement
- *No specific details provided on lateral movement beyond the endpoint compromise. The focus was on initial execution and data collection.*
### Data Exfiltration/Impact
- **Date/Time:** Occurring during the execution window (Dec 25 - Dec 30, 2022).
- **Details:** The malicious payload gathered system information and files, exfiltrating the findings via encrypted DNS queries directed toward `*.h4ck[.]cfd`.
### Detection & Response
- **Detection:** The mechanism of detection is not detailed, but awareness/disclosure occurred by December 31, 2022.
- **Response Actions:** The creator of the copied package deleted all collected data (as stated by the package creator). PyTorch likely released updated, trusted packages and advised users to audit their environments.
## Attack Methodology
- **Initial Access:** Package Dependency Confusion targeting the `torchtriton` dependency in PyTorch-nightly Linux builds.
- **Persistence:** Not specified, likely executed transiently upon installation/import.
- **Privilege Escalation:** Not specified, likely executing with user privileges granted during package installation.
- **Defense Evasion:** Use of encrypted DNS queries for exfiltration makes traffic inspection more difficult.
- **Credential Access:** Not explicitly mentioned.
- **Discovery:** The payload gathered "system information and files."
- **Lateral Movement:** Not specified.
- **Collection:** Gathering system information and local files.
- **Exfiltration:** Encrypted DNS queries.
- **Impact:** Unauthorized collection and transmission of sensitive system data.
## Impact Assessment
- **Financial:** Unknown/Not disclosed.
- **Data Breach:** System information and local files were accessed and exfiltrated from compromised Linux endpoints. The scope (number of users/sensitivity of data) is not fully quantified.
- **Operational:** Potential for temporary disruption as users might have needed to halt development or audit installations.
- **Reputational:** Damage to trust in the PyTorch package ecosystem, though the transparency of the official advisory helped mitigate long-term damage.
## Indicators of Compromise
- **Network Indicators (Defanged):** DNS queries directed to `h4ck[.]cfd` (or subdomains thereof).
- **File Indicators:** The execution of the specific malicious binary introduced via the `torchtriton` package installation. (Specific hash/filename not provided in context).
- **Behavioral Indicators:** Execution of arbitrary code during the installation/import phase of the PyTorch nightly package; unusual outbound encrypted DNS traffic originating from builds environments.
## Response Actions
- **Containment:** (Implied) Advising users to stop using the affected packages and potentially isolating affected systems.
- **Eradication:** (Implied) Removing the malicious package from public/internal indexes (if applicable) and building clean, signed repositories.
- **Recovery:** (Stated) The creator of the corrupted package deleted all collected data.
## Lessons Learned
- **Package Trust is Fragile:** Supply chain attacks, especially leveraging dependency confusion against build systems, are highly effective against widely used tools like Python's `pip`.
- **Ecosystem Vulnerability:** Relying on public repositories for dependency resolution without strict version pinning or source verification leaves systems exposed to name squatting.
## Recommendations
- Implement strict dependency resolution policies, including dependency pinning and requiring dependencies to be sourced from verified internal registries where possible, to mitigate dependency confusion risks.
- Utilize security tools that scan package metadata and source code for suspicious behavior before deployment in production or development environments.
- Regularly audit software bills of material (SBOMs) for newly introduced transient dependencies.