How to Detect and Remove Invisible Unicode Characters, ZWSP & Trojan Source Bidi Attacks
Invisible characters are one of the most frustrating sources of silent software bugs and emerging cybersecurity threats. When copying text from PDFs, chatbots, or chat apps, characters that occupy zero visual width on your screen get pasted into code, SQL queries, or configuration files.
In this guide, we explore the types of hidden characters, the security risks they pose, and how to scrub them with zero data exposure.
1. Types of Invisible Unicode Characters
- Zero-Width Space (U+200B) & Word Joiner (U+2060): Occupy no visual space but break string equality, causing
"admin" === "admin"to evaluate tofalse. - Byte Order Mark (U+FEFF): Often accidentally prepended to UTF-8 files, causing JSON parsers to throw unexpected token errors at line 1, column 1.
- Trojan Source BiDi Controls (U+202E, U+2066): Right-to-Left Override controls allow attackers to visually reverse code execution order, hiding malicious logic inside innocent-looking comments.
- Plane 14 Tag Prompt-Injection Payloads (U+E0001 - U+E007F): Hidden Unicode tags used by adversarial attackers to smuggle prompt-injection instructions past human reviewers into AI systems.
Clean Invisible Characters in Seconds
Detect zero-width spaces, normalize exotic whitespace, remove Trojan Source controls, and inspect hidden prompt-injection tags offline.
Open Invisible Character Cleaner →2. Preserving Legitimate Context: Emojis & Non-Latin Scripts
A naive regex that strips all zero-width characters destroys legitimate content:
- Emoji ZWJ Sequences: Complex emojis (like family groups, skin tones, or profession emojis) rely on the Zero-Width Joiner (U+200D) to glue base emojis together. Stripping it breaks emojis into individual disconnected glyphs.
- Persian & Indic Scripts: The Zero-Width Non-Joiner (U+200C) is an essential orthographic character in Persian (Farsi) and Hindi Devanagari script for cursive letter boundaries.
The MafLabs Invisible Character Cleaner uses context-aware regex engines to preserve authentic emoji and script sequences while aggressively purging rogue invisible artifacts.