How Intl.Segmenter Breaks Strings in JavaScript

The Intl.Segmenter API is a built-in JavaScript object that provides locale-sensitive text segmentation, enabling developers to split strings into meaningful units such as graphemes (characters), words, or sentences. Unlike standard string manipulation methods like split() or regular expressions, Intl.Segmenter respects language-specific boundary rules, making it an essential tool for properly handling multilingual text, complex emojis, and non-spaced languages such as Japanese, Chinese, or Thai.

Understanding the Syntax

To use the API, instantiate a new Intl.Segmenter instance by passing a locale identifier (or array of locales) and an options object defining the segmentation granularity.

const segmenter = new Intl.Segmenter(locale, { granularity: 'grapheme' | 'word' | 'sentence' });

The granularity property accepts three values: * 'grapheme': Splits text into individual visual characters (properly handling combined emojis and diacritics). * 'word': Splits text into words, punctuation, and whitespace boundaries. * 'sentence': Splits text into full sentences based on linguistic conventions.

Calling segmenter.segment(inputString) returns an iterable Segments object containing the segmented data.

Breaking Strings into Words

Splitting text by simple spaces using string.split(' ') fails in languages that do not use spaces and often leaves punctuation attached to words. The word granularity solves this by identifying semantic word boundaries and providing metadata to distinguish words from punctuation and whitespace.

Each segment returned in word mode contains: * segment: The extracted substring. * index: The zero-based character index where the segment begins. * input: The original source string. * isWordLike: A boolean flag indicating whether the segment is an actual word (true) or whitespace/punctuation (false).

Example: Extracting Words

const text = "Hello, world! How are you doing today?";
const segmenter = new Intl.Segmenter('en', { granularity: 'word' });

const segments = segmenter.segment(text);

// Filter out spaces and punctuation using isWordLike
const words = Array.from(segments)
  .filter(segment => segment.isWordLike)
  .map(segment => segment.segment);

console.log(words);
// Output: ["Hello", "world", "How", "are", "you", "doing", "today"]

Handling Non-Spaced Languages

Intl.Segmenter uses language dictionaries and break rules defined by the Unicode standard to split text accurately without spaces.

const japaneseText = "今日は良い天気ですね。";
const jpSegmenter = new Intl.Segmenter('ja', { granularity: 'word' });

const jpWords = Array.from(jpSegmenter.segment(japaneseText))
  .filter(s => s.isWordLike)
  .map(s => s.segment);

console.log(jpWords);
// Output: ["今日", "は", "良い", "天気", "です", "ね"]

Breaking Strings into Sentences

Splitting sentences with basic regular expressions often leads to errors when encountering abbreviations (e.g., “Dr.”, “e.g.”, “inc.”) or decimal numbers. The sentence granularity applies Unicode sentence boundary algorithms (UAX #29) tailored to the specified locale.

Example: Extracting Sentences

const paragraph = "Dr. Smith arrived at 6 a.m. He was very tired! Did he sleep well?";
const sentenceSegmenter = new Intl.Segmenter('en', { granularity: 'sentence' });

const sentenceSegments = sentenceSegmenter.segment(paragraph);

for (const { segment, index } of sentenceSegments) {
  console.log(`[Index ${index}]: ${segment.trim()}`);
}

// Output:
// [Index 0]: Dr. Smith arrived at 6 a.m.
// [Index 28]: He was very tired!
// [Index 47]: Did he sleep well?

Finding a Specific Segment by Index

The Segments object includes a .containing(index) method. This method allows you to retrieve the exact segment that contains a specified character position, which is useful for text editors, cursor navigation, and highlight tools.

const text = "The quick brown fox.";
const segmenter = new Intl.Segmenter('en', { granularity: 'word' });
const segments = segmenter.segment(text);

// Find the word under index 5 (inside the word "quick")
const currentSegment = segments.containing(5);

console.log(currentSegment);
// Output: { segment: "quick", index: 4, input: "The quick brown fox.", isWordLike: true }