Full Report
You should think twice before trusting your AI assistant, as database poisoning can markedly alter its output – even dangerously so
Analysis Summary
# Vulnerability: AI/ML Database Poisoning Attacks
## CVE Details
- CVE ID: Not specified in the provided text.
- CVSS Score: Not specified in the provided text.
- CWE: Not specified in the provided text, but relates to general data integrity and adversarial ML threats (e.g., CWE-172: Improper Neutralization of Data Injection for Machine Learning Model Inputs).
## Affected Systems
- Products: Generative AI systems, specifically Large Language Models (LLMs) and AI assistants.
- Versions: Not specified; the vulnerability is inherent to the training methodology and data handling processes of continuously updated ML/AI models.
- Configurations: Any AI/ML system built on continuously updated training datasets that are not fully verified or vetted.
## Vulnerability Description
Adversaries can perform **Data (or Database) Poisoning** attacks by injecting malicious or tainted data points into the training datasets used by AI/ML models (especially LLMs). This tampering alters the model's behavior, causing it to generate incorrect, biased, or dangerous outputs when triggered. Key attack vectors discussed include: Data Injection, Insider Attacks, Trigger Injection, and Supply-Chain Attacks on third-party model components.
## Exploitation
- Status: Discussion focuses on *potential* threats and documented historical examples (e.g., Tay Twitter bot incident). The text suggests these are known and insidious threats, but does not confirm widespread exploitation of current general-purpose LLMs under a specific CVE.
- Complexity: Varies by attack type. Trigger injection can be difficult to spot, while insider attacks exploit existing trust. Generally, data-level attacks require access to the training pipeline/datasets.
- Attack Vector: Generally related to data supply and training pipeline integrity. Can be considered **Network** (via external data sources/supply chain) or **Local** (via insider access).
## Impact
The impact is primarily on the reliability and safety of the AI output.
- Confidentiality: Potentially low, though model interactions might reveal sensitive data during training.
- Integrity: **High**. The core functionality and decision-making logic of the model are compromised.
- Availability: Medium to High, as trust in the system may be eroded, leading to decommissioning or required retraining.
## Remediation
### Patches
- No specific vendor patches are mentioned as this describes a class of security flaw in ML methodology. Remediation is procedural, focusing on securing the ML lifecycle.
### Workarounds
- **Constant checks and audits:** Continually validate the integrity and provenance of all training datasets.
- **Adversarial Training:** Guide models during supervised learning to recognize and reject malicious data points based on known attack signatures or anomalies.
- **Zero Trust and Access Management:** Implement strict authentication and authorization to monitor and restrict unauthorized access to core training data and the model update pipeline.
- **Security Focus in Development:** Adopt a prevention-first approach to minimize the attack surface during the model development lifecycle.
## Detection
- **Detection Methods:** Active monitoring for unusual data input patterns during training. Auditing dataset integrity before ingestion.
- **Indicators of Compromise:** Shifts in model output behavior that correlate with specific, previously injected inputs (triggers). Unexplained introduction of bias or harmful content in responses.
## References
- Vendor advisories: None specific mentioned.
- Relevant links:
- https://www(dot)welivesecurity(dot)com/2021/02/15/record-breaking-number-vulnerabilities-reported-2020/
- https://www(dot)welivesecurity(dot)com/2021/10/26/putting-cybersecurity-first-why-secure-by-design-must-be-norm/
- https://www(dot)salford(dot)ac(dot)uk/business/greater-manchester-cyber-foundry/cybersecurity-isnt-a-priority-for-smes-right-change-your-strategy
- https://www(dot)cbsnews(dot)com/news/microsoft-shuts-down-ai-chatbot-after-it-turned-into-racist-nazi/
- https://www(dot)sciencedirect(dot)com/science/article/pii/S0167404822002085
- https://www(dot)welivesecurity(dot)com/en/business-security/assessing-mitigating-cybersecurity-risks-supply-chain/
- https://www(dot)welivesecurity(dot)com/en/business-security/security-privacy-challenges-large-language-models/
- https://www(dot)eset(dot)com/int/business/protect-platform/
- https://www(dot)eset(dot)com/int/prevention-first-approach/
- https://www(dot)eset(dot)com/blog/enterprise/traveling-your-zero-trust-journey-with-eset/