New LLM Jailbreak "Immersive World" Proves AI Can Be Manipulated for Malware Creation

A new jailbreak technique, dubbed "Immersive World," has revealed weaknesses in the security controls of major Large Language Models (LLMs), enabling the creation of a fully functional Google Chrome infostealer. The technique, developed by a Cato CTRL threat intelligence researcher, bypasses LLM guardrails through narrative engineering, effectively tricking AI models into producing malware. The researcher, who had no prior malware development experience, successfully jailbroke multiple LLMs, including DeepSeek-R1, DeepSeek-V3, Microsoft Copilot, and OpenAI's ChatGPT-4o. According to Cato CTRL, "A Cato CTRL threat intelligence researcher with no prior malware coding experience successfully jailbroke multiple LLMs, including DeepSeek-R1, DeepSeek-V3, Microsoft Copilot, and OpenAI's ChatGPT-4o to create a fully functional Google Chrome infostealer for Chrome 133. An infostealer is malware that steals sensitive information—including login details, financial information, and other personally identifiable information (PII)."

The "Immersive World" jailbreak method operates by embedding LLMs into an elaborate fictional environment where cybersecurity restrictions are treated as non-existent. Within this framework, AI models are assigned specific roles, such as expert malware developers or security researchers, allowing them to produce malicious code under the guise of legitimate research. In controlled tests, the Cato researcher used this approach to refine an infostealer capable of extracting sensitive credentials from the Chrome Password Manager. The success of this technique demonstrates the severity of abuse with AI enabling non-technical users to generate sophisticated malware. In response to these findings, "Cato Networks reached out to DeepSeek, Microsoft, and OpenAI with its LLM jailbreak technique. DeepSeek was unresponsive. Microsoft and OpenAI acknowledged receipt. Cato Networks reached out to Google and offered to share the code of the Chrome infostealer. Google acknowledged receipt but declined to review the code," reports Cato.

The implications of this research extend beyond the immediate threat posed by AI-assisted malware development. As highlighted by Cato CTRL’s threat intelligence report, AI adoption has surged within corporate environments, with tools like Copilot, ChatGPT, Gemini (Google), Perplexity, and Claude (Anthropic) witnessing adoption increases ranging from 34% to 115% throughout 2024. This rising integration of AI models into business operations further amplifies the urgency of addressing jailbreak vulnerabilities. The research also raises concerns about the emergence of "zero-knowledge" threat actors—individuals without formal cybersecurity training who, by leveraging AI, can execute highly complex cyberattacks. As Cato Networks noted, "We believe the rise of the zero-knowledge threat actor poses a high risk to organizations because the barrier to creating malware is now substantially lowered with GenAI tools."