Full Report
In this special edition, we’ve selected the most-read Cybersecurity Snapshot items about AI security this year. ICYMI the first time around, check out this roundup of data points, tips and trends about secure AI deployment; shadow AI; AI threat detection; AI risks; AI governance; AI cybersecurity uses — and more.ICYMI, here are six things that’ll help you better understand AI security.1 - Best practices for secure AI system deploymentLooking for tips on how to roll out AI systems securely and responsibly? The guide “Deploying AI Systems Securely” has concrete recommendations for organizations setting up and operating AI systems on-premises or in private cloud environments. “Deploying AI systems securely requires careful setup and configuration that depends on the complexity of the AI system, the resources required (e.g., funding, technical expertise), and the infrastructure used (i.e., on premises, cloud, or hybrid),” reads the 11-page document, jointly published by cybersecurity agencies from the Five Eyes Alliance countries: Australia, Canada, New Zealand, the U.K. and the U.S.The agencies recommend that organizations developing and deploying AI systems incorporate the following: Ensure a secure deployment environment: Confirm that the organization’s IT infrastructure is robust, with good governance, a solid architecture and secure configurations in place. Require a threat model: Have the primary developer of the AI system — whether it’s a vendor or an in-house team — provide a threat model that can guide the deployment team in implementing security best practices, assessing threats and planning mitigations.Promote a collaborative culture: Encourage communication and collaboration among the organization’s data science, IT infrastructure and cybersecurity teams to address any risks or concerns effectively.Design a robust architecture: Implement security protections at the boundaries between the IT environment and the AI system; address identified blind spots; protect proprietary data sources; and apply secure design principles, including zero trust frameworks.Harden configurations: Follow best practices for the deployment environment, such as using hardened containers for running ML models; applying allowlists on firewalls; encrypting sensitive AI data; and employing strong authentication.For more information about deploying AI systems securely:“OWASP AI Security and Privacy Guide” (OWASP)“5 steps to make sure generative AI is secure AI” (Accenture)“Securing AI Makes for Safer AI” (CSET)“How to manage generative AI security risks in the enterprise” (TechTarget)“Security threats of AI large language models are mounting, spurring efforts to fix them” (Silicon Angle)2 - Dealing with the “shadow AI” problem As organizations scale up their AI adoption, they must closely monitor the usage of unapproved AI tools by employees — an issue known as “shadow AI.”So how do you identify, manage and prevent shadow AI? The Cloud Security Alliance’s “AI Organizational Responsibilities: Governance, Risk Management, Compliance and Cultural Aspects” white paper offers recommendations to tackle shadow AI, including:Creating a comprehensive inventory of AI systemsConducting gap analyses to spot discrepancies between approved and actual AI usageImplementing ways to detect unauthorized AI waresEstablishing effective access controlsDeploying monitoring techniques “By focusing on these key areas, organizations can significantly reduce the risks associated with shadow AI, ensuring that all AI systems align with organizational policies, security standards, and regulatory requirements,” the white paper reads.For example, to create an inventory that offers the required visibility into AI assets, the document explains different elements each record should have, such as:The asset’s descriptionInformation about its AI modelsInformation about its data sets and data sourcesInformation about the tools used for its development and deploymentDetailed documentation about its lifecycle, regulatory compliance, ethical considerations and adherence to industry standardsRecords of its access control mechanismsMeanwhile, the report “Oh, Behave! The Annual Cybersecurity Attitudes and Behaviors Report 2024-2025” from the National Cybersecurity Alliance (NCA) adds insights to the issue of employee use AI, with its finding that almost 40% of employees have fed sensitive work information to AI tools without their employers’ knowledgeThese findings, according to the NCA, highlight why organizations must urgently adopt AI usage policies and offer AI security training so employees understand the risks of using this technology.Have you ever shared sensitive work information without your employer’s knowledge?(Source: “Oh, Behave! The Annual Cybersecurity Attitudes and Behaviors Report 2024-2025” study by the National Cybersecurity Alliance, September 2024)For more information about AI risks:“4 Types of Gen AI Risk and How to Mitigate Them” (Harvard Business Review)“World leaders still need to wake up to AI risks” (Science Daily)“To understand the risks posed by AI, follow the money” (The Conversation)“Why can’t anyone agree on how dangerous AI will be?” (Vox)“Too Much Trust in AI Poses Unexpected Threats to the Scientific Process” (Scientific American)3 - How AI boosts real-time threat detectionAI has greatly impacted real-time threat detection by analyzing large datasets at unmatched speeds and identifying subtle, often-overlooked, changes in network traffic or user behavior. For example, AI can detect when a system atypically accesses sensitive data. Traditional tools may miss these nuanced anomalies, but AI systems are adept at spotting them.“For security, GenAI can revolutionize the field if applied correctly, especially when it comes to threat detection and response. It enhances efficiency and productivity by swiftly processing and delivering critical information when it matters most,” Nicholas Weeks, a Tenable senior product marketing manager, wrote in a blog post.One of AI's significant advantages in threat detection is its ability to be proactive. AI-powered systems continuously refine their algorithms as new malware strains and attack techniques emerge, learning from each event and integrating new insights into their threat detection mechanisms. This allows them to respond to both known and unknown threats more effectively than traditional, static, signature-based tools. "There has been automation in threat detection for a number of years, but we're also seeing more AI in general. We're seeing the large models and machine learning being applied at scale," Josh Schmidt, partner in charge of the cybersecurity assessment services team at BPM, a professional services firm, told TechTarget.In addition to monitoring internal network behavior, AI systems can more comprehensively analyze external sources of intelligence like RSS feeds, cybersecurity forums and global threat data. This wide-reaching capability helps AI gather actionable insights and recommend defense strategies that are tailored to current attack trends. For example, AI can flag a spike in phishing attacks targeting specific industries and suggest measures to counter these emerging threats. Additionally, as AI-generated phishing lures become nearly impossible for humans to detect, researchers and operators are turning to AI-based systems to assess if an email was AI-generated by looking for subtle telltales or differences when compared to a legitimate human-sourced email. For more information about ways in which AI can help cybersecurity teams:“GenAI Drives Broader Use of Artificial Intelligence Tech for Cyber” (Tenable)“Evaluate the risks and benefits of AI in cybersecurity” (TechTarget)“The role for AI in cybersecurity” (Cybersecurity Dive)“AI Is About To Take Cybersecurity By Storm” (Tenable)“New Generative AI Tools Aim to Improve Security” (Dark Reading)4 - New database aims to round up all AI risksFinding it hard to track all the cyber risks impacting AI systems? Check out the Massachusetts Institute of Technology’s AI Risk Repository, which aims to consolidate in a single place all risks associated with the use of artificial intelligence.To compile the database’s initial set of 700-plus risks, MIT analyzed 43 existing AI risk frameworks, and found that even the most comprehensive framework overlooked about 30% of all risks currently listed in the database.“Since the AI risk literature is scattered across peer-reviewed journals, preprints, and industry reports, and quite varied, I worry that decision-makers may unwittingly consult incomplete overviews, miss important concerns, and develop collective blind spots,” project leader and MIT postdoctoral researcher Peter Slattery said in a statement.The AI Risk Repository’s risk domains include:AI system safety, failures, and limitationsSocioeconomic and environmental harmsDiscrimination and toxicityPrivacy and securityMalicious actors and misuseThe risk domains are further subdivided into 23 subdomains. The AI Risk Repository is a “living database” that’ll be expanded and updated, according to MIT.Meanwhile, the January publication from the U.S. National Institute of Standards and Technology (NIST) “Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations (NIST.AI.100-2)” aims to help AI developers and users understand the types of attacks their AI systems can be vulnerable to, as well as ways to mitigate these threats.Specifically, the publication zeroes in on four attack types:Evasion attacks, which focus on altering an input to trick the AI system into responding erratically to it, such as tampering with a road stop sign to confuse an autonomous vehiclePoisoning attacks, in which corrupted data is fed to an AI system during its training phase, so that its output is erratic, inaccurate and inappropriatePrivacy attacks, which are launched during an AI system’s deployment and attempt to uncover confidential training data to then misuse the informationAbuse attacks, in which incorrect information is loaded into a legitimate but compromised source of data used by the AI systemTaxonomy of attacks on generative AI systems(Source: NIST’s “Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations (NIST.AI.100-2)” document, January 2024) For more information about protecting AI systems from cyberattacks:“Adversarial Machine Learning and Cybersecurity: Risks, Challenges, and Legal Implications” (Stanford University and Georgetown University)“Top 10 Critical Vulnerabilities for Large Language Model Applications” (OWASP)“Guidelines for secure AI system development” (U.S. and U.K. governments)“Vulnerability Disclosure and Management for AI/ML Systems” (Stanford University)“Adversarial AI Attacks Highlight Fundamental Security Issues” (Dark Reading)5 - Why cybersecurity pros are warming up to AI’s potentialA majority of cybersecurity professionals feel cautiously hopeful about artificial intelligence’s potential for strengthening their organizations’ cyber defenses, while also recognizing AI’s risks and adoption obstacles.That’s according to a global survey of almost 2,500 IT and security professionals conducted by the Cloud Security Alliance (CSA).“While there’s optimism about AI’s role in enhancing security, there’s also a clear recognition of its potential misuse and the challenges it brings,” reads the “State of AI and Security Survey Report,” which was commissioned by Google.Specifically, 63% of respondents said AI can potentially boost their organizations’ cybersecurity processes. Only 12% felt the opposite way. The rest had no opinion.Already, 22% of polled organizations use generative AI for security. More than half (55%) plan to use it within the next year, with the top use cases being rule creation, attack simulation and compliance monitoring. C-level and board support is driving generative AI adoption.Furthermore, 67% have tested AI for security purposes, and 48% feel either “very” or “reasonably” confident in their organizations’ ability to use AI for security successfully.What are your desired outcomes when it comes to implementing AI in your security team?(Source: Cloud Security Alliance’s “State of AI and Security Survey Report,” April 2024)Meanwhile, in a commissioned study conducted by Forrester Consulting on behalf of Tenable in October 2023, 44% of IT and security leaders polled said they were either “extremely confident” or “very confident” about their ability to use generative AI to enhance their organization’s cybersecurity strategy. In addition, 68% of respondents showed some level of interest in using GenAI to align IT/security goals with business goals; and a similar number — 67% — showed interest in using it to increase or improve the way their organization practices preventive cybersecurity. To get more details, check out the CSA report’s announcement “More Than Half of Organizations Plan to Adopt Artificial Intelligence (AI) Solutions in Coming Year” and the full 33-page report “State of AI and Security Survey Report.”For more information about how AI can help cybersecurity teams:“Generative AI making big impact on security pros, to no one’s surprise” (CSO)“How generative AI will enhance cybersecurity in a zero-trust world” (VentureBeat)“Envisioning Cyber Futures with AI” (Aspen Institute)“The Real-World Impact of AI on Cybersecurity Professionals” (ISC2)“6 ways generative AI chatbots and LLMs can enhance cybersecurity” (CSO)6 - A new GenAI governance guide for your org’s leadersHere’s a guide that might interest business and tech chiefs eager to ensure their organizations develop and deploy generative AI securely and responsibly.The Open Worldwide Application Security Project (OWASP) guide “LLM AI Cybersecurity & Governance Checklist” is aimed at business, privacy, compliance, legal and cybersecurity leaders, among others, tasked with setting guardrails for their organization’s generative AI use. The goal: Help them stay abreast of AI developments so that their organizations will reap business success from their generative AI use while avoiding legal, security and regulatory pitfalls.“These leaders and teams must create tactics to grab opportunities, combat challenges, and mitigate risks,” reads the document, which was created by the same OWASP team in charge of the group’s “OWASP Top 10 for LLM Applications” list.Areas covered by the checklist include:Adversarial riskThreat modelingAsset inventoryingSecurity and privacy trainingLegal and regulatory considerationsFor more information about using generative AI responsibly and securely:“Considerations for Implementing a Generative Artificial Intelligence Policy” (ISACA)“What every CEO should know about generative AI” (McKinsey & Co.)“A CISOs Guide: Generative AI and ChatGPT Enterprise Risks” (Team8)“Guidelines for secure AI system development” (U.S. and U.K. governments)“Security Implications of ChatGPT” (Cloud Security Alliance)
Analysis Summary
# Best Practices: AI Security
## Overview
These practices address the growing need to secure Artificial Intelligence (AI) systems, encompassing the entire lifecycle of AI models and applications to manage emerging risks and vulnerabilities.
## Key Recommendations
### Immediate Actions
1. **Apply Foundational Cybersecurity Principles:** Implement core security measures across all AI infrastructure, assuming AI systems are a critical component of the overall attack surface.
2. **Maintain Visibility Across the Attack Surface:** Utilize exposure management platforms to gain comprehensive visibility across IT, Cloud, OT/IoT, and Identity exposures that may impact AI deployments.
### Short-term Improvements (1-3 months)
1. **Integrate Security into AI Development Pipelines:** Integrate security checks (Security as Code) directly into Machine Learning Operations (MLOps) pipelines to catch vulnerabilities early.
2. **Assess Cloud-Native AI Exposure:** If utilizing cloud-based infrastructure for AI/ML workloads, implement Cloud Security Posture Management (CSPM) and Cloud Infrastructure Entitlement Management (CIEM) solutions to manage configuration drift and excessive permissions for AI resources.
3. **Conduct Prompt Engineering Audits:** Review and sanitize inputs (prompts) going into Generative AI models to prevent prompt injection attacks and data leakage.
### Long-term Strategy (3+ months)
1. **Establish Comprehensive AI Risk Quantification:** Develop metrics and use AI-specific analytics tools to accurately communicate cyber risk related to AI assets to business stakeholders.
2. **Implement Just-in-Time (JIT) Access for AI Environments:** Enforce JIT access controls for development, training, and deployment environments associated with high-value AI models to minimize standing privileges.
3. **Secure the Open Source Supply Chain for AI:** Implement rigorous scanning and management for open-source components used in building AI models (e.g., using tools focused on Open Source security) to mitigate supply chain risks.
## Implementation Guidance
### For Small Organizations
- Focus effort on **securing the environments where AI models are trained and hosted** (likely cloud or local VM/container infrastructure) by applying robust identity and access management (IAM) controls.
- Utilize **free or low-cost vulnerability scanning tools** (like Nessus Expert trials) to continuously assess the underlying infrastructure supporting the AI applications.
### For Medium Organizations
- Begin **integrating security testing early in the MLOps lifecycle** (Shift Left).
- Standardize on an **Exposure Management Platform** to correlate traditional IT vulnerabilities with the unique risks posed by cloud-based AI infrastructure components.
- Begin **formal training for developers** on secure AI/ML coding practices.
### For Large Enterprises
- Deploy a **full Exposure Management Platform** capable of providing unified visibility across IT, Cloud Exposure (CNAPP/CIEM), and OT/IoT layers, ensuring AI assets are mapped within this context.
- Establish dedicated **AI Threat Modeling** processes that examine poisoning, evasion, and inference attacks unique to the deployed models.
- Leverage **advanced reporting and business-contextual risk metrics** to guide executive decision-making regarding AI investments and security posture.
## Configuration Examples
*The provided context describes the need for security tools (like Tenable products for vulnerability/cloud/identity exposure) but does not contain specific technical configuration examples (e.g., firewall rules, IAM policies, or specific security settings for LLMs).*
**Guidance Placeholder:** Future steps in implementing these practices should include:
1. Configuring CSPM/CIEM solutions to monitor production environment IAM roles used by AI services.
2. Implementing input validation and sanitization libraries specifically designed to counteract common prompt injection vectors.
3. Establishing continuous monitoring thresholds for newly deployed AI model dependencies.
## Compliance Alignment
- **NIST Cybersecurity Framework (CSF):** Practices align broadly with Identify (asset management/risk assessment), Protect (access control, developing secure software), and Detect/Respond (continuous monitoring).
- **ISO 27001/27002:** Core principles of access control, secure system engineering, and supply chain security are foundational.
- **SLCGP Cybersecurity Plan Requirements:** Tenable specifically highlights that their solutions help fulfill SLCGP requirements, implying alignment with government standards for comprehensive risk management and exposure visibility.
## Common Pitfalls to Avoid
- **Treating AI as an Isolated System:** Ignoring the underlying infrastructure (cloud accounts, containers, operating systems) that hosts the AI/ML model is a major security gap.
- **Focusing Only on Model Output:** Overlooking training data integrity (data poisoning) and model inference security in favor of only checking the final application interface.
- **Delayed Security Scanning:** Waiting until models are deployed to scan for infrastructure vulnerabilities rather than embedding security checks into the MLOps pipeline from the start.
## Resources
- **Exposure Management Platform:** Utilize solutions offering unified visibility across traditional IT, Cloud, and Identity exposure (e.g., Tenable One).
- **Cloud Security Tools:** Implement Cloud Security Posture Management (CSPM) and Cloud Infrastructure Entitlement Management (CIEM) tools for AI/ML environments hosted in the cloud.
- **Vulnerability Management:** Employ continuous vulnerability scanning for underlying servers and containers hosting AI functions (e.g., Nessus products).
- **Education:** Enroll personnel in courses focused on securing these specific environments (e.g., Nessus Fundamentals Training references).