IcuDictionaryBreakEngine Class

The engines are ports of the ones of ICU (icu4c/source/common/dictbe.cpp), which the word break iterator and the line break iterator of ICU use, and so the browsers and Intl.Segmenter of JavaScript:

• IcuDictionaryBreakEngine.ChineseJapanese (CjkBreakEngine, without the phrase breaking): the segmentation of the run whose words have the smallest total cost. The run is normalized to NFKC; a character that does not start a word of one character is a word of the highest cost, and a run of katakana is a candidate word with a cost that depends on its length.
• IcuDictionaryBreakEngine.Thai, IcuDictionaryBreakEngine.Lao, IcuDictionaryBreakEngine.Khmer and IcuDictionaryBreakEngine.Burmese (ThaiBreakEngine, LaoBreakEngine, KhmerBreakEngine and BurmeseBreakEngine): the longest words of the list that are followed by other words of the list, with up to three words of lookahead. The characters that start no word are attached to the word before them, up to a plausible word start; a break is never before a combining mark, and Thai attaches the elision and repetition characters to the word before them.

The engines do not find the runs: a caller applies the rules of its segmentation (the word or line breaking rules of Unicode, of ICU, or of CSS) and gives each run of the characters an engine handles (IcuDictionaryBreakEngine.Handles) to that engine. The text is read by code point, an unpaired surrogate being a code point, as ICU reads it. The engines are stateless and can be used by several threads at the same time; the word list of an engine is loaded on its first use.