Cybersecurity51 views6 min read

Microsoft Detects Phishing Attack Using AI-Generated Code

Microsoft has detected and blocked a phishing campaign that used AI-generated code to hide its malicious payload within an SVG file using business terminology.

Leo Martinez
By
Leo Martinez

Leo Martinez is a cybersecurity correspondent for Neurozzio, focusing on threat intelligence, malware analysis, and emerging digital security risks. He translates complex technical threats for a broad audience.

Author Profile
Microsoft Detects Phishing Attack Using AI-Generated Code

Microsoft Threat Intelligence has identified and stopped a credential phishing campaign that employed a novel obfuscation technique, likely generated by a large language model (LLM). The attackers hid malicious code within a Scalable Vector Graphics (SVG) file, using business-related terminology to evade standard security detection systems.

The campaign, which primarily targeted organizations in the United States, highlights a growing trend of cybercriminals experimenting with artificial intelligence to improve the effectiveness of their attacks. Despite the sophisticated disguise, Microsoft's own AI-powered defenses successfully blocked the threat.

Key Takeaways

  • A phishing campaign on August 18 used a compromised email account to send malicious SVG files disguised as PDF documents.
  • Attackers likely used a large language model (LLM) to generate complex, obfuscated code that mimicked legitimate business data.
  • The malicious code was hidden using business terms like "revenue" and "operations" within an invisible part of the SVG file.
  • Microsoft's Security Copilot assessed the code as synthetic, noting its complexity and structure were not typical of human programming.
  • The attack was successfully detected and blocked by Microsoft Defender for Office 365, which analyzed behavioral and infrastructural signals.

Details of the Phishing Campaign

On August 18, Microsoft’s security teams detected a phishing operation that originated from a compromised small business email account. The attackers used a common tactic of self-addressing the email, placing the actual targets in the Blind Carbon Copy (BCC) field to bypass simple detection rules.

The email itself was designed to look like a standard file-sharing notification. It contained a simple message prompting the recipient to open an attachment. The attached file was named 23mb – PDF- 6 pages.svg, a name intended to mislead users into thinking it was a safe PDF document.

The Role of SVG Files in Attacks

Scalable Vector Graphics (SVG) files are an increasingly popular choice for cybercriminals. Unlike standard image files, SVGs are text-based and can contain executable scripts, such as JavaScript. This allows attackers to embed malicious payloads directly into what appears to be a simple image.

According to Microsoft's analysis, SVGs offer several advantages for attackers. They can use features like invisible elements and encoded attributes to hide malicious content from both users and many automated security tools. This makes them an effective vector for delivering interactive phishing attacks.

How the Attack Unfolded

When a user opened the malicious SVG file, it would redirect their browser to a webpage. This page presented a CAPTCHA challenge, a common social engineering tactic used to create a false sense of security. After the user completed the CAPTCHA, the campaign would have likely displayed a fake sign-in page to capture their credentials. Microsoft's systems blocked the attack before this final stage was observed.

A Novel Obfuscation Method

The most distinctive feature of this campaign was its method of hiding the malicious code. Instead of using traditional encryption, the attackers used business-related language as a form of camouflage. This was achieved in two primary ways.

First, the SVG file's code was structured to appear as a business analytics dashboard. It included elements for a "Business Performance Dashboard" with chart bars and month labels. However, these elements were made completely invisible by setting their opacity to zero, serving only as a decoy for anyone inspecting the file's source code.

Second, the actual malicious instructions were encoded using a long string of business terms. Words such as revenue, operations, risk, and shares were embedded within a hidden attribute of an invisible text element. An embedded JavaScript would then process these words, converting them back into executable code that initiated the phishing attack.

From Business Terms to Malicious Code

The embedded script was designed to systematically decode the sequence of business terms. It mapped pairs or sequences of these words to specific characters or instructions. This process reconstructed the hidden functions, which included redirecting the user's browser, collecting browser fingerprint data, and starting a session tracking mechanism.

AI Analysis Confirms Synthetic Code

Given the unusual and complex nature of the obfuscation, Microsoft researchers suspected the involvement of artificial intelligence. They used Microsoft Security Copilot to analyze the SVG file's contents to determine if it was machine-generated.

The analysis concluded that it was highly likely the code was synthetic and generated by an LLM. Security Copilot identified several indicators pointing to AI generation rather than human programming.

The code was “not something a human would typically write from scratch due to its complexity, verbosity, and lack of practical utility.”

Key indicators identified by the security tool included:

  • Overly Descriptive Naming: Function and variable names followed a pattern of descriptive words followed by random hexadecimal strings (e.g., processBusinessMetricsf43e08). This is a common trait of AI-generated code.
  • Over-Engineered Structure: The code was highly modular and repetitive, a characteristic of AI models that tend to generalize solutions.
  • Generic Comments: The comments were verbose and used formal business language, such as “Advanced business intelligence data processor,” which is a hallmark of AI-generated documentation.
  • Formulaic Obfuscation: The techniques used to hide the code were implemented in a thorough but formulaic manner, consistent with AI code generation.

Defending Against AI-Enhanced Threats

While attackers are beginning to use AI to create more sophisticated attacks, Microsoft emphasizes that these threats are not undetectable. AI-generated code still operates within the same behavioral and infrastructural boundaries as human-written attacks. Security systems that analyze a wide range of signals can effectively counter these evolving tactics.

Microsoft Defender for Office 365 successfully blocked this campaign by analyzing signals beyond the payload itself. These signals included:

  • Email Delivery Patterns: The use of self-addressed emails with BCC recipients was a major red flag.
  • Suspicious File Type: The use of an SVG file named to look like a PDF was identified as atypical for legitimate document sharing.
  • Malicious Infrastructure: The SVG payload redirected to a domain already flagged for phishing activity.
  • Code Obfuscation: The mere presence of obfuscation, regardless of its novelty, was an indicator of malicious intent.

Microsoft noted that AI-generated code often introduces new, synthetic artifacts that can themselves become signals for detection. The verbose naming conventions and redundant logic seen in this campaign are examples of such artifacts.

Recommendations for Organizations

To defend against emerging threats, Microsoft Threat Intelligence recommends several best practices. These measures are effective against a broad range of phishing attacks, including those that may use AI-generated code.

Organizations should configure email security settings to recheck links at the time of click and enable features that retroactively quarantine malicious messages. Users should be encouraged to use modern web browsers with built-in protection against malicious websites. Finally, implementing phishing-resistant authentication methods, such as those available through Microsoft Entra, can significantly strengthen an organization's security posture.