Full Report
State-backed threat actors from a handful of countries are using ChatGPT for a range of malicious purposes ranging from malware refinement to employment scams and social media disinformation campaigns.
Analysis Summary
# Threat Actor: Nation-State and Cybercrime Groups Exploiting ChatGPT
## Attribution & Identity
Attribution is linked to state-backed threat actors from **China, Russia, North Korea, Iran, and the Philippines**, as well as suspected crime groups operating out of **Cambodia**.
Known Chinese APT Groups explicitly mentioned:
* **APT5** (also known as Keyhole Panda)
* **APT15** (also known as Vixen Panda)
## Activity Summary
A range of threat actors utilized OpenAI's ChatGPT for illicit purposes, categorized into three main buckets: social media disinformation, cyberattack assistance/malware refinement, and foreign employment scams.
**Influence Operations (Primarily China):**
* Dozens of China-based accounts used ChatGPT to bulk generate social media posts, often seeking responses in English based on Chinese prompts.
* Content focused on controversial topics, including the shutdown of USAID, U.S. political discourse, backlash against Taiwan, and criticism of China's investments in Balochistan (mentioning activist Mahrang Baloch).
* Generated comments in English, Chinese, and Urdu, posted across TikTok, X, Reddit, and Facebook, sometimes simulating engagement between multiple accounts.
* **Russia:** Generated German-language content criticizing NATO and the U.S. for German federal elections on X and Telegram.
* **Iran:** Used the tool for social media comment generation covering geopolitical topics.
* **Philippines:** Used to mass-generate comments supporting President Bongbong Marcos's policies.
**Malware Refinement & Cyberattack Assistance:**
* **China-affiliated actors (APT5, APT15):** Sought assistance with brute-forcing passwords, writing scripts for multi-factor login attempts, scanning servers for specific ports, AI-driven penetration testing, and generating code for social media automation. They inquired about U.S. defense industry, U.S. military networks, and government technology (e.g., Special Operations Command, satellite communications, ground station locations).
* **Russia:** Used ChatGPT to develop and refine Windows malware named **ScopeCreep** (used against video game players), debug code, and set up C2 infrastructure. They operated stealthily using temporary email addresses and limiting each account to incremental code improvements.
**Employment Scams:**
* **Cambodia-based groups:** Focused on generating recruitment messages in numerous languages (English, Spanish, Swahili, Kinyarwanda, German, Haitian Creole) to lure victims into the cyber scam industry with promises of high salaries for trivial tasks (e.g., liking social media posts). They also sought translations from Chinese into multiple languages.
* **North Korea:** Allegedly used ChatGPT to research technical tools to circumvent corporate security measures and maintain a persistent, undetected remote presence as part of an IT worker scheme.
## Tactics, Techniques & Procedures
* Bulk generation of social media comments/posts for influence operations.
* Script generation for brute-forcing passwords and testing login combinations.
* Code debugging and malware development assistance (Windows malware).
* Researching specific technical details for penetration testing and infrastructure profiling.
* Using AI to simulate user interaction on social media platforms (actors possessed pre-existing code for Android device simulation).
* Mass-generating recruitment messages in multiple languages for human trafficking/scam operations.
## Targeting
* **Sectors:** U.S. defense industry, U.S. military, video game players, IT/corporate environments (implied by NK activity).
* **Geography:** U.S. (targeted inquiries), Germany (election commentary), Balochistan region (political targeting), Global social media platforms (TikTok, X, Reddit, Facebook). Victims of scams originate from various regions globally, trafficked to Cambodia.
* **Victims:** Specific organizations targeted by inquiry were those related to the U.S. Special Operations Command and defense contractors.
## Tools & Infrastructure
* **Malware families used:** **ScopeCreep** (Russian-attributed Windows malware).
* **Infrastructure:** C2 setup assistance sought/developed for ScopeCreep; use of temporary email addresses for ChatGPT account creation (Russian actors); execution via Android simulation code.
## Implications
The primary implication is the widespread adoption of generative AI tools like ChatGPT across the threat landscape for espionage, financial crime, and information warfare, even by established APTs. While OpenAI stated the AI did not provide *novel* capabilities, it significantly accelerates and streamlines existing workflows, such as malware iteration, social media influence operations, and initial research phases for targeting highly sensitive sectors like U.S. defense. The involvement of state actors from China, Russia, and North Korea in code refinement and targeting sensitive infrastructure confirms AI integration into mature cyber espionage programs.
## Mitigations
* Heightened monitoring for rapid iteration of code, unusual script requests, or detailed inquiries regarding critical infrastructure (especially military/defense).
* Vigilance against employment scams leveraging low-barrier recruitment tactics in multiple languages (particularly those involving high salary promises for simple tasks).
* Security teams should assume threat actors are using AI tools to refine existing malware (like ScopeCreep) and anticipate faster development cycles.
* Monitoring social media platforms for sudden increases in coordinated influence messaging, especially those targeting geopolitical fault lines.