Lodash words: Custom Regex Pattern Matching

The _.words function in Lodash splits a string into an array of isolated character blocks, operating either through built-in lexical parsers or via custom regular expressions. When supplied with a custom regex, _.words bypasses its default ASCII and Unicode boundary algorithms and directly applies the specified pattern to extract matched segments. This article explains the internal mechanics behind dynamic pattern processing in _.words, how the method isolates custom character blocks, and key implementation considerations.

The Core Architecture of _.words

At its source level, _.words accepts two arguments: the target string and an optional pattern:

_.words([string=''], [pattern])

When the pattern argument is omitted or undefined, Lodash inspects the input string using an internal utility (hasUnicodeWord) to determine whether it contains complex Unicode symbols, emojis, or specific script combinations. Based on that check, it routes the string to either asciiWords or unicodeWords, which apply predefined regular expressions designed to split standard phrases, camelCase, and kebab-case tokens.

However, when a custom pattern is provided, Lodash bypasses both internal tokenizers completely. The internal implementation simply delegates execution to the native JavaScript engine:

function words(string, pattern) {
  if (pattern === undefined) {
    return hasUnicodeWord(string) ? unicodeWords(string) : asciiWords(string);
  }
  return string.match(pattern) || [];
}

If the pattern produces no matches, _.words safely returns an empty array rather than null.

Dynamic Character Block Isolation

Because _.words feeds the custom pattern directly into String.prototype.match, character block isolation is determined entirely by the regex expression passed to it. This allows developers to define custom token boundaries that ignore standard whitespace or punctuation conventions.

1. Custom Delimiters and Segment Extraction

Standard word splitting treats symbols like colons, hyphens, and slashes as delimiters. By passing a custom regex, those symbols can either serve as explicit boundaries or become part of the isolated blocks themselves:

const path = "src/components/button.jsx";

// Isolate path segments ignoring file extensions
const segments = _.words(path, /[^/.]+/g);
// Output: ['src', 'components', 'button', 'jsx']

2. Isolating Specific Lexical Tokens

Custom patterns allow for domain-specific token extraction, such as extracting hashtags, mentions, or specific code tokens without post-processing:

const post = "Deploying #v2 with @team for #launch!";

// Isolate only hashtags including the symbol
const hashtags = _.words(post, /#[a-zA-Z0-9]+/g);
// Output: ['#v2', '#launch']

The Role of the Global Flag (g)

When passing a custom regular expression to _.words, the presence of the global flag (g) is critical due to how String.prototype.match() behaves in JavaScript:

• With the g flag: The engine evaluates the entire string and returns an array of all matching character blocks.
• Without the g flag: The engine returns an array containing only the first complete match along with its capturing groups, index, and input references.

To dynamically extract all character blocks matching a pattern across an entire string, the supplied RegExp must include the global flag:

const text = "item1, item2, item3";

// Without 'g': only captures the first instance
_.words(text, /item\d/);
// Output: ['item1']

// With 'g': captures every matching block
_.words(text, /item\d/g);
// Output: ['item1', 'item2', 'item3']

Performance and Predictability

By delegating directly to String.prototype.match, _.words minimizes Lodash overhead when a custom pattern is used. It avoids the performance cost of scanning for Unicode grapheme clusters and running multiple fallback regular expressions. The native engine isolates the requested character blocks in a single pass, making custom regex matching through _.words both computationally efficient and fully customizable.