Full Report
[…] a real example of an AWS Kubernetes cluster infection through a software development supply chain compromise. The attackers were able to get AWS credentials from a DevOps workstation and use them to introduce a poisoned docker image into a Kubernetes cluster. It allowed th...
Analysis Summary
# Incident Report: AWS Kubernetes Supply Chain Compromise via Poisoned Docker Image
## Executive Summary
This incident involved a sophisticated attack targeting an AWS Kubernetes (K8s) cluster via a software development supply chain compromise. Attackers successfully compromised a DevOps workstation, obtained AWS credentials, and injected a malicious Docker image into the K8s environment. This resulted in the retrieval of sensitive cloud secrets and tokens, though the malicious activity was detected just in time to prevent further lateral movement to other organizations or execution of more severe payloads.
## Incident Details
- Discovery Date: **Undisclosed (Detected just in time)**
- Incident Date: **Undisclosed (Prior to May 25, 2023)**
- Affected Organization: **Undisclosed**
- Sector: **Technology/Cloud Services (Implied by target technologies)**
- Geography: **Undisclosed**
## Timeline of Events
### Initial Access
- Date/Time: **Undisclosed**
- Vector: **End-user compromise leading to workstation compromise.**
- Details: Attackers compromised a **DevOps workstation**, likely gaining access to the internal network or development environment credentials.
### Lateral Movement
- Date/Time: **Undisclosed**
- Vector: **Supply Chain Injection (Docker/Kubernetes)**
- Details: The compromised credentials were used to introduce a **poisoned Docker image** into the **Kubernetes cluster**. Once running within the cluster, the attackers moved laterally within the cluster environment and further into the associated **AWS cloud environment**.
### Data Exfiltration/Impact
- Date/Time: **Undisclosed**
- Vector: **Cloud Credential Harvesting**
- Details: Attackers **retrieved secrets, passwords, and tokens** from the AWS environment. This data could have facilitated movement to other companies or execution of more damaging payloads.
### Detection & Response
- Date/Time: **Undisclosed (Detected "just in time")**
- Vector: **Monitoring/Alerting (Implied)**
- Details: The malicious activity was detected by security monitoring just as the attackers harvested sensitive secrets, preventing the planned next stages of the attack. (Specific response actions are detailed in the Response Actions section).
## Attack Methodology
- Initial Access: **End-user Compromise (DevOps Workstation)**
- Persistence: **Poisoned Docker Image within K8s Cluster**
- Privilege Escalation: **Utilization of stolen AWS credentials**
- Defense Evasion: **Undisclosed (Implied by successful injection)**
- Credential Access: **AWS credentials obtained from the compromised DevOps workstation.**
- Discovery: **Undisclosed (Likely performed post-K8s access to map cloud resources)**
- Lateral Movement: **Within the Kubernetes cluster, followed by lateral movement into the connected AWS environment.**
- Collection: **Secrets, passwords, and tokens from the cloud environment.**
- Exfiltration: **Implied, as credentials were retrieved before detection.**
- Impact: **Unauthorized access to cloud credentials and potential for major resource compromise.**
## Impact Assessment
- Financial: **Undisclosed**
- Data Breach: **Sensitive credentials, passwords, and tokens exposed.**
- Operational: **Near-miss scenario; potential for significant operational disruption averted by timely detection.**
- Reputational: **Undisclosed**
## Indicators of Compromise
- **Network indicators:** N/A (No specific IPs/URLs provided)
- **File indicators:** Malicious/Poisoned Docker Image Artifacts
- **Behavioral indicators:** Execution of containers attempting to access AWS metadata or secret stores; Unusual retrieval of cloud configuration secrets.
## Response Actions
- **Containment measures:** Undisclosed, but likely involved immediate isolation/deletion of the compromised Docker image/K8s deployment and revocation/rotation of all retrieved AWS credentials.
- **Eradication steps:** Undisclosed, focused on cleaning repositories and scanning build pipelines.
- **Recovery actions:** Undisclosed, but must have included hardening the DevOps workstation and K8s access controls.
## Lessons Learned
- **Key takeaways:** Software supply chain compromises (specifically poisoned container images) are a high-impact vector, even when initial access is through a standard workstation compromise. Cloud credentials stored or accessible on development/DevOps workstations present a critical risk.
- **What could have been done better:** Improved security hardening on the DevOps workstation, stricter image signing and verification within the Kubernetes admission controllers, and stronger Multi-Factor Authentication (MFA) or least-privilege access to AWS resources.
## Recommendations
- Implement **Mandatory Image Signing and Verification** for all images deployed to the Kubernetes cluster (e.g., using Notary or similar solutions).
- **Harden DevOps Workstations:** Implement stricter controls, application whitelisting, and endpoint detection and response (EDR) on systems used for primary software delivery, especially those holding cloud credentials.
- **Zero Trust for Cloud Access:** Implement **short-lived credentials**, utilize IAM roles over long-term keys wherever possible, and enforce MFA for all console and programmatic access to AWS.
- **Scanning:** Integrate vulnerability and malware scanning into the CI/CD pipeline, specifically targeting base images and layers within the Docker build process before promotion to the registry.