Lodash Unicode Normalization Gaps and Limits

Lodash is one of the most widely adopted JavaScript utility libraries, but its string parsing suite relies on custom regular expressions and basic character mappings rather than standard Unicode specifications. Consequently, Lodash natively lacks implementation for standard Unicode Normalization Forms (NFC, NFD, NFKD, and NFKC), compatibility decompositions, comprehensive script deburring, and full grapheme cluster segmentation. This article outlines the specific Unicode normalization strategies that Lodash lacks natively and details the technical limitations of its built-in string functions.

Absence of Unicode Normalization Forms (UAX #15)

Lodash does not natively implement or invoke any of the four standard Unicode Normalization Forms defined by Unicode Standard Annex #15:

• NFC (Canonical Decomposition, followed by Canonical Composition): Lodash cannot canonically combine base characters and adjacent combining diacritics into precomposed characters.
• NFD (Canonical Decomposition): Lodash does not decompose precomposed characters into their base and combining components standardly.
• NFKC (Compatibility Decomposition, followed by Canonical Composition): Lodash provides no native way to resolve compatibility formatting while recomposing canonical characters.
• NFKD (Compatibility Decomposition): Lodash fails to decompose compatibility characters into canonical equivalents.

Instead of leveraging JavaScript's native String.prototype.normalize(), functions like _.kebabCase, _.camelCase, _.words, and _.deburr rely on hardcoded ASCII/Latin-based regular expressions.

Lack of Compatibility Equivalence (NFKD/NFKC Transformations)

Because Lodash lacks compatibility normalization strategies, it cannot parse or standardize characters that convey identical semantic meaning but differ typographically. Specifically, Lodash lacks:

• Ligature Decomposition: Characters such as "fi" (U+FB01) or "fl" (U+FB02) are treated as single unknown characters rather than decomposing into "fi" and "fl".
• Width Normalization: Fullwidth and halfwidth forms (such as Fullwidth Latin characters like "Ａ" U+FF21) are not mapped to their ASCII equivalents ("A").
• Enclosed and Styled Alphanumerics: Circled digits (e.g., "①"), font-styled math letters (e.g., "𝕭"), and superscripts/subscripts (e.g., "²", "₅") are not normalized to their base forms.
• Fraction Normalization: Vulgar fractions (e.g., "½" U+00BD) are not decomposed into separate digits and slash characters.

Latin-Restricted Diacritic Removal in _.deburr

Lodash provides _.deburr to strip diacritics, which many casing functions use internally. However, its normalization strategy is limited to:

1. A manual mapping dictionary covering only Latin-1 Supplement and Latin Extended-A characters.
2. A regular expression that strips a narrow range of combining diacritical marks ([\u0300-\u036f\ufe20-\ufe2f\u20d0-\u20ff]).

Because of this design, Lodash lacks normalization for:

• Non-Latin Scripts: Diacritics and accents in Greek (e.g., polytonic accents), Cyrillic, Hebrew, Arabic, and Indic scripts are completely ignored.
• Complex Latin Extensions: Characters from Latin Extended-B, Latin Extended Additional (commonly found in Vietnamese, such as "ệ" or "ơ"), and phonetic extensions are largely omitted from decomposition unless pre-composed into combining marks matching its targeted range.

Missing Full Unicode Case Folding (UAX #21)

Functions such as _.toLower, _.lowerCase, _.toUpper, and _.upperCase delegate to JavaScript's standard toLowerCase() and toUpperCase() without implementing full Unicode Case Folding. This leaves several gaps:

• Special One-to-Many Casing: Handling of characters like the German lowercase sharp S ("ß") is not normalized to "SS" across all contexts or matched symmetrically against uppercase strings.
• Case Folding Equivalence: Case normalization does not account for folding strategies that align characters such as the Greek uppercase sigma ("Σ"), lowercase sigma ("σ"), and final sigma ("ς") into a single canonical comparison point.

Absence of Standard Grapheme Cluster Segmentation (UAX #29)

Lodash functions that divide strings—such as _.words, _.truncate, and _.split—rely on internal regular expressions designed to capture basic surrogate pairs, but they do not implement modern Extended Grapheme Cluster normalization strategies. As a result, Lodash fails to parse:

• Zero-Width Joiner (ZWJ) Sequences: Complex emojis (such as family groups or professional roles) are split into individual components rather than treated as single visual units.
• Skin-Tone and Hair Modifiers: Unicode modifier sequences are separated from their base characters during operations like truncation or word extraction.
• Regional Indicator Pairs: Flag sequences made from pairs of Regional Indicator symbols can be fractured improperly.