Look-alike characters

edit

Many characters look like others: the Cyrillic а looks like the Latin a. An attacker can register раураl, written with Cyrillic letters, to pass for paypal. UTS #39 defines how to detect such names, and Unicode implements it.

Detect names that look alike

Unicode.AreConfusable tells whether two strings look alike:

Assert.True(Unicode.AreConfusable("paypal", "\u0440\u0430\u0443\u0440\u0430l")); // Cyrillic look-alikes
Assert.True(Unicode.AreConfusable("caf\u00E9", "cafe\u0301")); // composed vs decomposed
Assert.True(Unicode.AreConfusable("paypal", "pay\u200Bpal")); // invisible characters are ignored
Assert.False(Unicode.AreConfusable("paypal", "example"));

To find the collisions in a set of names, such as the user names of an application, store the skeleton of each name and look new names up by their skeleton. Unicode.GetConfusableSkeleton gives it:

var userName = "\u0440\u0430\u0443\u0440\u0430l";
var skeleton = Unicode.GetConfusableSkeleton(userName);

Assert.Equal(Unicode.GetConfusableSkeleton("paypal"), skeleton);

The skeleton is a comparison key, not text: do not show it to users, and do not store it in place of the name.
Two strings are confusable when their skeletons are equal. The comparison does not consider the scripts: combine it with the detection of mixed scripts below.
The skeleton uses the normalization of this package, not the one of the platform: it is the same everywhere, including in globalization-invariant mode.
A string with an unpaired surrogate throws an ArgumentException.

Detect mixed scripts

A spoofed name usually mixes writing systems. Unicode.IsMixedScript applies the resolved script set rules of UTS #39, and Unicode.IsSingleScript is its opposite:

Assert.False(Unicode.IsMixedScript("paypal")); // all Latin
Assert.True(Unicode.IsMixedScript("\u0440\u0430\u0443\u0440\u0430l")); // Cyrillic letters with a Latin "l"
Assert.False(Unicode.IsMixedScript("日本語です")); // Japanese legitimately mixes Han and Hiragana
Assert.False(Unicode.IsMixedScript("user123")); // digits match any script

Digits, punctuation and combining marks match any script: they never make a string mixed on their own.
Japanese and Korean text, which mixes Han with Hiragana, Katakana or Hangul, is not mixed.
The empty string is single-script.

Replace look-alike characters

Unicode.ReplaceConfusablesCharacters replaces each character that has a look-alike by its prototype, and Unicode.IsConfusableCharacter tells whether a character has one:

var input = "\u0440\u0430\u0443\u0440\u0430l"; // "раураl": Cyrillic letters that look like Latin ones
var normalized = Unicode.ReplaceConfusablesCharacters(input);

Assert.Equal("paypal", normalized);

To compare strings, prefer AreConfusable: it also normalizes the strings, so that a composed and a decomposed accent compare equal.