1. The Vulnerability of Global Script Diversity
When the Unicode Consortium unified the world's scripts, it preserved authentic glyph shapes for every historical and modern writing system. However, Latin, Greek, Cyrillic, and Cherokee alphabets share common Phoenician ancestral roots.
As a result, dozens of characters in distinct scripts are visually indistinguishable to the human eye, even though they possess completely different Unicode code points. These characters are known as Homoglyphs.
| Visible Glyph | Latin Character | Lookalike Homoglyph |
|---|---|---|
| a | U+0061 (Latin Small Letter A) | U+0430 (Cyrillic Small Letter A) |
| e | U+0065 (Latin Small Letter E) | U+0435 (Cyrillic Small Letter Ie) |
| o | U+006F (Latin Small Letter O) | U+03BF (Greek Small Letter Omicron) |
| p | U+0070 (Latin Small Letter P) | U+0440 (Cyrillic Small Letter Er) |
2. The Attack Vectors
Vector 1: IDN Homograph Phishing
In 2005, security researcher Evgeniy Gabrilovich demonstrated that an attacker could register apple.com where the letter 'a' was replaced by the Cyrillic U+0430. In a browser address bar, the domain looked 100% identical to the authentic site.
To combat this, the Internet Assigned Numbers Authority (IANA) introduced Punycode (RFC 3492). If an address bar detects mixed scripts in a single domain label, it renders the raw ASCII representation prefixed with xn-- (e.g. xn--pple-43d.com).
Vector 2: Trojan Source (CVE-2021-42574)
In 2021, researchers revealed that Unicode Bidirectional (BiDi) override controls (such as U+202E Right-to-Left Override) can reorder source code visually in IDEs while the compiler processes the code in raw byte order:
3. Unicode Normalization Forms (NFC, NFD, NFKC, NFKD)
Many characters in Unicode can be encoded in more than one way. For instance, the letter 'é' can be encoded as:
- Precomposed: A single code point
U+00E9(Latin Small Letter E with Acute) - Decomposed: Two separate code points
U+0065(Letter 'e') +U+0301(Combining Acute Accent)
To human eyes, they render identically. But to a database query:
"\u00E9" === "\u0065\u0301" // Returns FALSE in JavaScript and SQL!
If a user creates an account with precomposed 'é' and attempts to log in with decomposed 'é', authentication fails unless the backend performs Unicode Normalization:
| Form | Name | Mechanism |
|---|---|---|
| NFC | Canonical Decomposition, followed by Canonical Composition | Standardizes characters into their single precomposed form. (Web standard recommended by W3C). |
| NFD | Canonical Decomposition | Splits characters into base letters followed by combining accents (used in macOS HFS+ filesystem). |
| NFKC | Compatibility Decomposition, followed by Composition | Replaces stylistic variants (e.g. superscript ², ligature fi, circled ①) with plain standard equivalents. |
| NFKD | Compatibility Decomposition | Splits stylistic characters into their raw plain letters and separate diacritics. |
4. Developer Best Practices for String Sanitization
The Security Checklist for Web Engineers:
- Always normalize incoming usernames and passwords using
string.normalize('NFKC')before hashing or querying databases. - Enforce single-script validation for critical identifiers: reject usernames containing both Latin and Cyrillic/Greek characters in the same token.
- Lint code repositories with CI/CD tools that block BiDi control characters (U+202A through U+202E).
Test text normalization and homoglyph detection
Try our interactive Homoglyphs Scanner and Normalization Inspector.