Ctrl+Alt+Deceit: An Update on AI-Based Cyber Attacks
Close to three years after ChatGPT ignited mainstream generative-AI adoption, the offensive side of the security equation is making huge strides. When I first talked about the Era of the Autonomous Defense and the inevitability of Proudly Offensive security, my focus was on how defenders could automate at machine-speed. Since then, real-world vulnerabilities and breaches have shown that attackers are weaponizing the very same tooling. My partner Jess Leão just dropped a must-read on models that happily yank their own kill-switches, blackmail operators, and scheme around oversight for example. Given the rapid developments I wanted to pull together a short field report on some of the emerging styles of AI-enabled attacks, together with the implications each style creates.
Prompt-Injection and Tool-Chain Hijacks
LLMs wired into developer workflows are now a direct path to source-code theft and supply-chain compromise. Last month, Researchers at Legit Security showed that a single base-encoded comment inside a merge request could coerce GitLab Duo (powered by Claude) to dump private repositories, inject rogue HTML and phone home to an attacker-controlled URL.
In a very similar case, Invariant Labs uncovered a critical flaw in the widely-used GitHub MCP server, starred 14 k+ times on GitHub, that lets a malicious Issue trick an AI agent into leaking private repos. The way it works is that the adversary plants a public-repo issue laced with a hidden prompt-injection string. When a developer’s AI assistant, Claude Desktop connected via the GitHub MCP server, pulls the list of open issues, it ingests the booby-trapped text, executes the rogue prompt, and cascades into a malicious agent workflow.
The same pattern hit the orchestration tier: CVE-2025-3248 in the open-source builder Langflow lets an unauthenticated attacker hit the /api/v1/validate/code endpoint and execute arbitrary Python NVD. CISA shoved the bug into its Known Exploited Vulnerabilities catalogue after detecting in-the-wild abuse, and internet scans still found hundreds of exposed servers days later. Once an LLM or low-code workflow gains exec() rights, a stealth prompt or crafted payload can exfiltrate more data in seconds than months of slow, covert theft.
Synthetic Impersonation and Deep-Fake Social Engineering
AI voice and video cloning have demolished the cost of credibility. In May, the FBI warned that adversaries were texting, then phoning, targets with AI-generated voices of senior U.S. officials to harvest credentials. The alert followed an earlier bureau bulletin on AI-driven vishing and smishing campaigns. Data backs the trend: CrowdStrike’s 2025 Global Threat Report logged a 442 % jump in vishing attacks between H2 2024 and H1 2025, fueled by AI voice cloning. Beyond reputational damage, every deep-fake call burns executive time, and one breached mailbox can ripple through partner ecosystems, creating a hidden productivity and trust tax rarely itemized in breach-cost spreadsheets.
Crime-ware-as-a-Service: Malicious LLMs and Phishing Factories
Generative AI is turning script-kiddies into enterprise-grade criminals. As one of our security founders likes to say “What was once nation state is now common place”. Dark-web operators now tout an uncensored chatbot dubbed FraudGPT for $200 per month or $1,700 per year, boasting 3,000+ confirmed sales that bundle turnkey malware and spear-phish generation. Delivery scales just as aggressively: Barracuda telemetry logged more than one million Phishing-as-a-Service attacks in January–through February 2025. 89% of them launched via the Tycoon 2FA kit, with EvilProxy and Sneaky 2FA rounding out the field. When anyone with $200 and a Telegram handle can mint flawless, localized phishing lures, the old “look for bad grammar” user-training tip is becoming more and more obsolete.
AI & IT Infrastructure Attack Vectors
Application-layer exploits are only half the story. Last week, Palisade Research’s red-teamers showed OpenAI’s o3 model disabling its own shutdown routine. If a model can jailbreak itself in the lab, the scaffolding around it is living on borrowed time. In its threat report published in February, OpenAI had reported that it had begun purging ChatGPT accounts tied to Chinese, Iranian and North-Korean state actors after discovering they used the service to debug spyware and craft influence campaigns.
As adversarial campaigns become more sophisticated it is likely it will extend into traditional network and endpoint infrastructure as well. In December of last year, Palo Alto Networks’ Unit 42 had published research that LLMs excelled at transforming existing malicious code into natural-looking, evasive variants, especially in JavaScript. Over time, these transformations can degrade the accuracy of malware classifiers, subtly training them to misidentify threats as benign. Damien Lewke had a great post on how to effectively seek these out by applying behavior, network, automation, artifact and entropy-based hunts, matched with prevention controls.
Conclusion
Adversaries have moved from experimenting with AI to operationalizing it at industrial scale. AI-enabled attacks have moved from one-off proofs of concept to systemic campaigns that breach developer pipelines, con executives with deep-fake voices, and mass-produce phishing kits. Simultaneously, adversaries are trying to find gaps in the AI and IT infrastructure plumbing itself, proving that defenses must extend past applications and down into the platforms that serve them. More than ever these styles of attacks are becoming embedded, pervasive and go beyond code to quite lucrative “as-a-service” businesses. In such a dynamically changing infrastructure and application environment, I am certain we will see even more varieties and styles of attacks. As always if you are building in AI and cybersecurity, please reach out to me!
Great article and regarding weaponising the AI is hitting the mainstream. The fact that some hackers can inject malicious code into a prompt is a whole new game.