CONNECT WITH US
AI & Deeptech

AI & Deeptech

Microsoft built a prompt injection detector. Then it caught a phishing campaign instead.

TheNewStack (LLM) - ArtificialIntelligence logo

Published on

Add as a preferred source on Google
Microsoft built a prompt injection detector. Then it caught a phishing campaign instead.

Microsoft flagged a phishing campaign last week that exploits a gap in how machines read text. Attackers are slipping invisible Unicode tag characters into email bodies; they don’t render on screen but change the underlying string that software processes.

The company says attackers are already using the technique at scale to bypass spam filters and ML-based classifiers, and the same approach could cause problems for AI systems that regularly ingest text from external sources.

Security researchers have documented a nearly identical technique targeting LLMs, commonly called “ASCII Smuggling,” which uses Unicode tag characters in the U+E0000 to U+E007F range — code points that exist in the character stream but aren’t displayed by most interfaces.

That gives you two versions of the same text: what a person reads and what software receives.

In the campaign tracked by Microsoft Defender for Office 365, attackers weren’t using tag characters to smuggle hidden instructions into an AI model. They placed them inside high-signal words associated with financial phishing, such as “funding,” “loan,” and “credit,” so that filters scanning for those terms would no longer find an exact match.

A hunting signature for ASCII Smuggling fired on roughly 21,000 messages the day before the campaign started and the next day, it fired on more than 1.3 million. Then, just two days later, the count passed 2.3 million. The whole time, recipients saw ordinary-looking offers for business loans and credit lines.

NLP systems break text into tokens before processing it, and slipping an unexpected Unicode character into a word can change how those tokens are formed.


Source link

Disclaimer

We strive to uphold the highest ethical standards in all of our reporting and coverage. We TheMorningPulse.fyi want to be transparent with our readers about any potential conflicts of interest that may arise in our work. It's possible that some of the investors we feature may have connections to other businesses, including competitors or companies we write about. However, we want to assure our readers that this will not have any impact on the integrity or impartiality of our reporting. We are committed to delivering accurate, unbiased news and information to our audience, and we will continue to uphold our ethics and principles in all of our work. Thank you for your trust and support.