Full Report
Kong Ingress Controller is a popular ingress controller for Kubernetes. The Kong Ingress Controller version 3.4 instances have been experiencing a significant performance regression causing excessive CPU utilization of approximately 4 cores, even with minimal Gateway API reso...
Analysis Summary
# Incident Report: Compromised Kong Ingress Controller Image Leading to Resource Hijacking
## Executive Summary
Unknown actors successfully compromised the build process for the Kong Ingress Controller version 3.4, injecting malicious code into the official Docker image published on DockerHub. This resulted in a supply chain attack leading to significant performance degradation (excessive CPU utilization) for deploying organizations, indicative of resource hijacking, likely for cryptojacking operations. The compromise was detected as a performance regression and has since been addressed by the maintainers.
## Incident Details
- **Discovery Date:** Late December 2024 / Early January 2025 (Based on initial reports of performance issues being widespread)
- **Incident Date:** Prior to December 2024 (The compromised image was available for over a week)
- **Affected Organization:** Kong (Vendor) and all utilizing organizations running KIC v3.4
- **Sector:** Cloud Native / Technology (Supply Chain Target)
- **Geography:** Global (Impacted consumers of DockerHub images)
## Timeline of Events
### Initial Access
- **Date/Time:** Not specified, predating the public identification of the issue.
- **Vector:** CI/CD Pipeline Compromise (Likely due to previously reported vulnerabilities in their chain).
- **Details:** Attackers leveraged a weakness in the Kong Ingress Controller's Continuous Integration/Continuous Delivery (CI/CD) workflow to inject malicious code into the resulting Docker image artifacts.
### Lateral Movement
- **Details:** Evidence suggests the attack was highly targeted at the build artifact itself (The Docker Image). Lateral movement within consumer environments is implied by the execution of the malicious script post-deployment, but the primary breach vector was the software supply chain.
### Data Exfiltration/Impact
- **Details:** The primary impact observed was **Resource Hijacking** evidenced by excessive CPU utilization (approx. 4 cores per instance). While not explicitly stated, this pattern strongly suggests cryptojacking. No specific data exfiltration was detailed for this incident.
### Detection & Response
- **Detection:** The issue manifested as a significant performance regression within consuming Kubernetes clusters (excessive CPU use even with minimal load).
- **Response Actions:** The Kong team addressed the issue, presumably by rolling back the compromised image tags or pushing a patched version.
## Attack Methodology
- **Initial Access:** Compromise of the vendor’s CI/CD pipeline.
- **Persistence:** Code inserted into the official Docker image, ensuring persistence as long as the compromised image tag (v3.4) was in use.
- **Privilege Escalation:** Not applicable in the traditional sense, but the artifact leveraged the permissions granted to the running Ingress Controller within the Kubernetes cluster.
- **Defense Evasion:** The payload was embedded directly within a trusted vendor artifact, bypassing traditional network/endpoint security controls focusing on ingress traffic.
- **Credential Access:** Not explicitly detailed.
- **Discovery:** Not explicitly detailed (relevant to the attacker's actions within victim environments).
- **Lateral Movement:** Not the focus; the attack focused on artifact injection.
- **Collection:** Not explicitly detailed.
- **Exfiltration:** Suspected cryptojacking operations imply CPU cycles/compute resources were the "stolen" asset.
- **Impact:** Performance degradation, resource hijacking, and supply chain compromise.
## Impact Assessment
- **Financial:** Unspecified, but significant operational costs due to wasted compute resources (4 CPU cores per instance).
- **Data Breach:** No customer data breach confirmed based on the provided context.
- **Operational:** Significant performance degradation/instability for any organization using the compromised KIC v3.4 image.
- **Reputational:** Negative impact on trust in vendor build processes and software supply chain security.
## Indicators of Compromise
- **Network Indicators (Defanged):** Likely outbound connections to known mining pools (if cryptojacking occurred), though none are listed.
- **File Indicators:** Malicious script injection observed within the running container processes or image layers (specific file hashes/names not provided).
- **Behavioral Indicators:** Excessive CPU utilization (approx. 4 cores) on Kubernetes worker nodes hosting the Kong Ingress Controller v3.4 pod, inconsistent with deployment configuration. Script injection observed during CI/CD verification steps for other users.
## Response Actions
- **Containment:** Consumers needed to immediately stop using or pull newer, clean images published by Kong. Kong likely revoked the compromised image tags.
- **Eradication:** Deleting/replacing all deployed instances of the compromised image.
- **Recovery:** Re-deploying the service using a verified, patched version of the Kong Ingress Controller.
## Lessons Learned
- The reliance on automated build pipelines (CI/CD) introduces a critical, high-leverage supply chain attack surface.
- Vulnerabilities in the build chain can have immediate, widespread operational impact before security teams can react to application-level exploits.
- Performance regressions can be a leading indicator of background resource hijacking (cryptojacking).
## Recommendations
- Implement mandatory immutable infrastructure practices to verify base images against upstream sources where possible.
- Harden CI/CD environments using least privilege principles and separation of duties.
- Utilize software bill of materials (SBOM) tools to verify container contents against expected compositions before deployment.
- Establish clear baseline metrics for core components (like Ingress Controllers) to quickly spot resource anomalies indicative of compromise.