Configurable spoof checker wrapping an opaque handle and optionally wrapping a previously serialized instance.
Moduletext-icu-0.8.0.5Haskell98
Data.Text.ICU.Spoof
String spoofing (confusability) checks for Unicode, implemented as bindings to the International Components for Unicode (ICU) uspoof library.
See UTR #36 and UTS #39 for detailed information about the underlying algorithms and databases used by this module.
- 6 types
- 13 values
- Packagetext-icu-0.8.0.5
- Exports19
- LanguageHaskell98
- LicenceBSD-3-Clause
- SourceSpoof.hsc
Unicode spoof checking API
6 declarationsThe spoofCheck, areConfusable, and getSkeleton functions analyze Unicode text for visually confusable (or "spoof") characters.
For example, Latin, Cyrillic, and Greek all contain unique Unicode values which appear nearly identical on-screen:
A 0041 LATIN CAPITAL LETTER A
Α 0391 GREEK CAPITAL LETTER ALPHA
А 0410 CYRILLIC CAPITAL LETTER A
Ꭺ 13AA CHEROKEE LETTER GO
ᴀ 1D00 LATIN LETTER SMALL CAPITAL A
ᗅ 15C5 CANADIAN SYLLABICS CARRIER GHO
A FF21 FULLWIDTH LATIN CAPITAL LETTER A
𐊠 102A0 CARIAN LETTER A
𝐀 1D400 MATHEMATICAL BOLD CAPITAL A
and so on. To check a string for visually confusable characters:
optionally configure it with setChecks, setRestrictionLevel, and/or setAllowedLocales, then
spoofCheck a single string, use areConfusable to check if two strings could be confused for each other, or use getSkeleton to precompute a "skeleton" string (similar to a hash code) which can be cached and re-used to quickly check (using Unicode string comparison) if two strings are confusable.
By default, these methods will use ICU's bundled copy of confusables.txt and confusablesWholeScript.txt, which could be out of date. To provide your own confusables databases, use openFromSource. (To avoid repeatedly parsing these databases, you can then serialize your configured MSpoof and later openFromSerialized to load the pre-parsed databases.)
Exception thrown with openFromSource fails to parse one of the input files.
Constructors
OpenFromSourceParseErrorerrFile :: OpenFromSourceParseErrorFileThe file which could not be parsed.
parseError :: ParseErrorParse error encountered opening a spoof checker from source.
Instances3Show, Exception, NFData
Show OpenFromSourceParseErrorDefined in text-icu-0.8.0.5 · Data.Text.ICU.SpoofException OpenFromSourceParseErrorDefined in text-icu-0.8.0.5 · Data.Text.ICU.SpoofNFData OpenFromSourceParseErrorDefined in text-icu-0.8.0.5 · Data.Text.ICU.Spoof
Constructors
SingleScriptConfusableMakes areConfusable report if both identifiers are both from the same script and are visually confusable. Does not affect spoofCheck.
MixedScriptConfusableMakes areConfusable report if both identifiers are visually confusable and at least one identifier contains characters from more than one script.
Makes spoofCheck report if the identifier contains multiple scripts, and is confusable with some other identifier in a single script.
WholeScriptConfusableMakes areConfusable report if each identifier is of a different single script, and the identifiers are visually confusable.
AnyCaseBy default, spoof checks assume the strings have been processed through
toCaseFoldand only check lower-case identifiers. If this is set, spoof checks will check both upper and lower case identifiers.RestrictionLevelChecks that identifiers are no looser than the specified level passed to setRestrictionLevel.
InvisibleChecks the identifier for the presence of invisible characters, such as zero-width spaces, or character sequences that are likely not to display, such as multiple occurrences of the same non-spacing mark.
CharLimitChecks whether the identifier contains only characters from a specified set (for example, via setAllowedLocales).
MixedNumbersChecks that the identifier contains numbers from only a single script.
AllChecksEnables all checks.
AuxInfoEnables returning a RestrictionLevel in the SpoofCheckResult.
Instances5Bounded, Enum, Eq, Show, ToBitMask
Bounded SpoofCheckDefined in text-icu-0.8.0.5 · Data.Text.ICU.SpoofEnum SpoofCheckDefined in text-icu-0.8.0.5 · Data.Text.ICU.SpoofEq SpoofCheckDefined in text-icu-0.8.0.5 · Data.Text.ICU.SpoofShow SpoofCheckDefined in text-icu-0.8.0.5 · Data.Text.ICU.SpoofToBitMask SpoofCheckDefined in text-icu-0.8.0.5 · Data.Text.ICU.Spoof
Constructors
CheckOKThe string passed all configured spoof checks.
CheckFailed [SpoofCheck]The string failed one or more spoof checks.
CheckFailedWithRestrictionLevelThe string failed one or more spoof checks, and failed to pass the configured restriction level.
failedChecks :: [SpoofCheck]The spoof checks which the string failed.
failedLevel :: RestrictionLevelThe restriction level which the string failed to pass.
Instances2Eq, Show
Eq SpoofCheckResultDefined in text-icu-0.8.0.5 · Data.Text.ICU.SpoofShow SpoofCheckResultDefined in text-icu-0.8.0.5 · Data.Text.ICU.Spoof
Constructors
ASCIIChecks that the string contains only Unicode values in the range ߝ inclusive.
SingleScriptRestrictiveChecks that the string contains only characters from a single script.
HighlyRestrictiveChecks that the string contains only characters from a single script, or from the combinations (Latin + Han + Hiragana + Katakana), (Latin + Han + Bopomofo), or (Latin + Han + Hangul).
ModeratelyRestrictiveChecks that the string contains only characters from the combinations (Latin + Cyrillic + Greek + Cherokee), (Latin + Han + Hiragana + Katakana), (Latin + Han + Bopomofo), or (Latin + Han + Hangul).
MinimallyRestrictiveAllows arbitrary mixtures of scripts.
UnrestrictiveAllows any valid identifiers, including characters outside of the Identifier Profile.
Instances5Bounded, Enum, Eq, Show, ToBitMask
Bounded RestrictionLevelDefined in text-icu-0.8.0.5 · Data.Text.ICU.SpoofEnum RestrictionLevelDefined in text-icu-0.8.0.5 · Data.Text.ICU.SpoofEq RestrictionLevelDefined in text-icu-0.8.0.5 · Data.Text.ICU.SpoofShow RestrictionLevelDefined in text-icu-0.8.0.5 · Data.Text.ICU.SpoofToBitMask RestrictionLevelDefined in text-icu-0.8.0.5 · Data.Text.ICU.Spoof
Constructors
SkeletonSingleScriptBy default, getSkeleton builds skeletons which catch visually confusable characters across multiple scripts. Pass this flag to override that behavior and build skeletons which catch visually confusable characters across single scripts.
SkeletonAnyCaseBy default, getSkeleton assumes the input string has already been passed through
toCaseFoldand is lower-case. Pass this flag to override that behavior and allow upper and lower-case strings.
Instances5Bounded, Enum, Eq, Show, ToBitMask
Bounded SkeletonTypeOverrideDefined in text-icu-0.8.0.5 · Data.Text.ICU.SpoofEnum SkeletonTypeOverrideDefined in text-icu-0.8.0.5 · Data.Text.ICU.SpoofEq SkeletonTypeOverrideDefined in text-icu-0.8.0.5 · Data.Text.ICU.SpoofShow SkeletonTypeOverrideDefined in text-icu-0.8.0.5 · Data.Text.ICU.SpoofToBitMask SkeletonTypeOverrideDefined in text-icu-0.8.0.5 · Data.Text.ICU.Spoof
Functions
13 declarationsOpen a spoof checker for checking Unicode strings for lookalike security issues with default options (all SpoofChecks except CharLimit).
Open a spoof checker with custom rules given the UTF-8 encoded
contents of the confusables.txt and confusablesWholeScript.txt
files as described in Unicode UAX #39.
Generates re-usable "skeleton" strings which can be used (via Unicode equality) to check if an identifier is confusable with some large set of existing identifiers.
If you cache the returned strings in storage, you must invalidate your cache any time the underlying confusables database changes (i.e., on ICU upgrade).
By default, assumes all input strings have been passed through
toCaseFold and are lower-case. To change this, pass
SkeletonAnyCase.
By default, builds skeletons which catch visually confusable characters across multiple scripts. Pass SkeletonSingleScript to override that behavior and build skeletons which catch visually confusable characters across single scripts.
Get the checks performed by a spoof checker.
Configure the checks performed by a spoof checker.
Get the restriction level of a spoof checker.
Configure the restriction level of a spoof checker.
Get the list of locale names allowed to be used with a spoof checker.
(We don't use LocaleName since the root and default locales have no
meaning here.)
Get the list of locale names allowed to be used with a spoof checker.
(We don't use LocaleName since the root and default locales have no
meaning here.)
Check if two strings could be confused with each other.
Checks if a string could be confused with any other.
Serializes the rules in this spoof checker to a byte array, suitable for re-use by openFromSerialized.
Only includes any data provided to openFromSource. Does not include any other state or configuration.