Full Report
A pair of AI tools advertised on hacking forums were developed using commercial AI models from xAI and Mistral, according to Cato Networks. The post Researchers say AI hacking tools sold online were powered by Grok, Mixtral appeared first on CyberScoop.
Analysis Summary
# Tool/Technique: WormGPT Variants (Powered by Grok and Mixtral)
## Overview
This refers to multiple variants of malicious Large Language Models (LLMs) marketed on underground forums (like BreachForums) as "WormGPTs." These tools are designed to bypass safety guardrails present in commercial AI models, allowing them to generate offensive content, code, and instructions for malicious activities. Specific variants analyzed were found to be powered by underlying models from **xAI's Grok** and **Mistral AI's Mixtral**.
## Technical Details
- Type: Tool (Malicious LLM/AI Agent)
- Platform: Software/Cloud-based interaction (via Telegram or web interfaces on forums)
- Capabilities: Generate phishing emails, code credential-stealing malware (e.g., PowerShell scripts), search for and analyze vulnerabilities, and provide hacking instructions.
- First Seen: The "WormGPT" concept was first advertised in 2023; these specific Grok and Mixtral powered variants were observed over the past year (as of June 2025).
## MITRE ATT&CK Mapping
Since these are AI tools used to facilitate attacks, the mapping reflects the *intent* and *output* of the resulting cyberattack, rather than the tool itself having a direct execution technique.
- **TA0001 - Initial Access** (For phishing content generation)
- **T1566 - Phishing**
- T1566.001 - Spearphishing Attachment
- T1566.002 - Spearphishing Link
- **TA0002 - Execution** (For generating malicious code)
- **T1059 - Command and Scripting Interpreter** (If PowerShell scripts are generated)
- **TA0003 - Persistence** (Potential for generating persistence mechanisms)
- **TA0007 - Discovery** (For vulnerability analysis assistance)
## Functionality
### Core Capabilities
- Generating offensive content, including phishing emails.
- Circumventing standard safety restrictions imposed by commercial LLM providers.
- Providing instructions related to cyberattacks.
### Advanced Features
- One variant was identified as a wrapper around **xAI's Grok**, using a system prompt modification to instruct Grok to bypass its own guardrails and produce malicious content.
- The other variant analyzed was found to be powered by **Mistral AI's Mixtral**.
- Capability to generate functional malware code, such as PowerShell scripts for credential theft.
## Indicators of Compromise
*Note: As these are AI models sold/accessed online, traditional malware IoCs (hashes, file names) are not applicable to the core sales mechanism. The IoCs relate to the *output* they generate.*
- File Hashes: N/A (Tool distribution method)
- File Names: N/A
- Registry Keys: N/A
- Network Indicators: Access likely occurred via specific **BreachForums** threads and **Telegram** channels associated with the sellers. (Specific URLs/IPs not provided by the context for defanging).
- Behavioral Indicators: Use of LLMs to generate high-fidelity, context-aware phishing content or request code for known malware families (e.g., PowerShell credential stealers).
## Associated Threat Actors
- Unnamed actors marketing and selling these uncensored AI models on underground hacking forums (e.g., BreachForums).
- General cybercriminal ecosystem seeking easier, more effective initial access and development tools.
## Detection Methods
- Detection must focus on the *usage* patterns rather than signatures of the model itself.
- **Behavioral Detection:** Monitoring platforms for queries that explicitly request the generation of malware code, exploitation steps, or bypassing established safety protocols (Jailbreaking attempts).
- **Network Detection:** Monitoring access to known underground forums and associated Telegram channels if network egress monitoring is possible.
## Mitigation Strategies
- **Vendor Awareness:** AI/LLM providers (like Mistral AI, xAI) must continuously monitor for and rapidly patch jailbreaking vulnerabilities that allow system prompts to be extracted or overwritten.
- **User Training:** Educating personnel on identifying sophisticated, AI-generated phishing content (which can be highly personalized).
- **Output Vetting:** Implementing strict code review processes for any automatically generated script or code snippet intended for operational use, assuming an AI source could introduce subtle flaws or malicious intent.
## Related Tools/Techniques
- WormGPT (Original concept)
- FraudGPT (Another noted predecessor/competitor)
- General LLM Jailbreaking techniques (used to access the underlying system prompts).