Full Report
LLM-enabled malware poses new challenges for detection. SentinelLABS presents groundbreaking research on how to hunt for this new class of threats.
Analysis Summary
# Research: Prompts as Code & Embedded Keys | The Hunt for LLM-Enabled Malware
## Metadata
- Authors: Alex Delamotte, Vitaly Kamluk, Gabriel Bernadett-Shapiro
- Institution: SentinelOne
- Publication: SentinelOne Blog (Abridged version of a LABScon 2025 presentation)
- Date: September 19, 2025
## Abstract
This research investigates the emerging threat of **LLM-enabled malware**, where malicious logic is intended to be generated or executed dynamically via Large Language Models (LLMs) rather than being statically embedded in the binary. The authors sought to define this new category of malware, assess its characteristics, and develop reliable hunting methodologies despite the potential for runtime code generation to evade traditional static analysis. A key finding was the discovery of samples leveraging embedded API keys and specific prompt structures as indicators, leading to the identification of "MalTerminal," potentially the earliest known example of such malware.
## Research Objective
The primary objective was to understand how LLMs are being used in the context of malware, specifically focusing on the category where LLM capabilities are embedded within the malicious payload to provide an operational advantage. The research aimed to:
1. Define "LLM-enabled" malware and differentiate it from classical malware.
2. Characterize its principal features and capabilities.
3. Develop reliable hunting strategies for novel or unknown samples.
4. Assess potential adversarial adaptations to increase robustness.
## Methodology
### Approach
The research employed a survey and threat hunting approach, focusing specifically on malware that "**Leverages LLM Capabilities**" as an embedded component. The methodology relied heavily on **pattern matching** against known indicators associated with LLM interaction, such as specific prompt structures and the presence of embedded API keys, which serve as proxies for dynamic malicious logic.
### Dataset/Environment
The study focused on analyzing malware samples in the wild that exhibited characteristics suggesting integration with external LLM services. The discovery process included identifying hitherto unknown samples, including one dubbed 'MalTerminal'. The research also uncovered other offensive LLM applications, such as red team utilities and code vulnerability injection tools.
### Tools & Technologies
The specifics of the scanning and analysis tools were not detailed, but the methodology heavily implies the use of **static analysis** (for pattern matching API keys and prompts) and potentially some form of **dynamic analysis** adapted to capture interactions with external APIs.
## Key Findings
### Primary Results
1. **Detection Gap:** LLM-enabled malware presents a significant challenge to conventional security tools because malicious logic can be generated at runtime, potentially bypassing static signatures and complicating dynamic analysis due to unpredictable behavior.
2. **Hunting Vector:** The research successfully identified LLM-enabled malware by pattern matching against static artifacts, specifically **embedded API keys** and **distinct prompt structures** used to direct the LLM.
3. **Discovery of 'MalTerminal':** The methodology led to the discovery of novel samples, including 'MalTerminal,' which the researchers suggest may be the earliest known example of malware leveraging embedded LLM functionality.
4. **Broader Misuse:** The hunt uncovered other LLM-based offensive tools, including people search agents, red team benchmarking utilities, and LLM-assisted code vulnerability injection tools.
### Supporting Evidence
The success of the hunting methodology validates the utility of using **embedded API keys** and **prompt structures** as reliable indicators of software attempting to interface with an LLM for malicious purposes.
### Novel Contributions
The primary contribution is the establishment of a focused methodology for **hunting LLM-enabled malware** by signature-matching for the necessary infrastructure (keys and prompts) required for this class of threat, rather than attempting to predict the volatile output code. The identification of the 'MalTerminal' sample provides an early real-world anchor point for this emerging malware type.
## Technical Details
The core technical challenge lies in the LLM's ability to generate code "on-the-fly." The research circumvents this by targeting the pre-LLM interface components. **Prompts as Code** refers to the specialized, often obfuscated or structured, instructions embedded within the malware that command the remote LLM to produce the desired malicious execution path. **Embedded Keys** refer to the presence of API credentials within the binary, linking the execution to a specific, controllable LLM endpoint.
## Practical Implications
### For Security Practitioners
Traditional signature-based detection methods are insufficient against malware whose core logic is derived dynamically from an LLM prompt. Practitioners must update threat model assessments to account for runtime generation of malicious payloads.
### For Defenders
Defensive strategy should pivot to **network egress monitoring** (to detect calls to LLM APIs from endpoints) and **static analysis focused on known LLM interaction indicators** (e.g., scanning binaries for patterns resembling API keys meant for AI services or common prompt formats used for generating specific offensive actions).
### For Researchers
This work stimulates further research into developing robust, LLM-aware detection techniques that can analyze the structure and intent of prompts being fed to models, even when those prompts are fragmented or obfuscated within a binary.
## Limitations
The research implicitly acknowledges the limitation that actors employing this technique are aware of signature-based discovery and may adapt by:
1. Using API endpoint routing to obscure true destinations.
2. Employing advanced obfuscation techniques for prompts and keys.
3. Relying on open-source or locally hosted, unmonitored LLMs.
## Comparison to Prior Work
Prior work often focused on: 1) LLMs as lures (social engineering); 2) Attacks against LLM-integrated systems (prompt injection); or 3) LLMs as external assistant tools (e.g., WormGPT). This research specifically isolates and targets the category where **the LLM functionality is integrated as a functional component within the malware payload itself**, which presents a unique detection challenge compared to these other adversarial uses.
## Future Work
Future work should focus on detecting polymorphic, LLM-generated code dynamically and investigating methods to characterize the *intent* encoded in the prompts themselves to build pre-execution behavior predictors.
## References
- [Previous SentinelOne analysis on Prompt Injection Vulnerabilities in Code Agents (External Link Reference)]
- [Publicly available research on LLM-generated malware feasibility (e.g., AITROJAN example) (External Link Reference)]
- [Research on adversarial misuse of Generative AI for operational support (External Link Reference)]