Hyphenation

Hyphenate a word

The rules of Unicode say where a line can break, but hyphenating a word needs data for its language. Hyphenator applies the pattern algorithm of TeX (Franklin Liang) to patterns you provide:

// The patterns, as plain text: "hy3ph he2n ..." or a \patterns{...} group, then exceptions such as "as-so-ciate".
// Use Hyphenator.Load("hyph-en-us.pat.txt", "hyph-en-us.hyp.txt") to read them from files.
var hyphenator = Hyphenator.Parse("hy3ph he2n hena4 hen5at 1na n2at 1tio 2io", "as-so-ciate");

Span<int> positions = stackalloc int[32];
var count = hyphenator.Hyphenate("hyphenation", positions);
Assert.Equal<int>([2, 6], positions[..count]); // hy-phen-ation

• Hyphenate takes one word, in any case, and writes the indexes before which a hyphen can be inserted. It does not allocate. Use Unicode.EnumerateWordBoundaries to find the words of a text (see Text segmentation).
• A break is never inside a grapheme cluster.
• The patterns match code points: pass the word in the normalization form of the patterns, usually NFC (see Normalization).
• A hyphenator is immutable and can be used by several threads at the same time: load it once per language and reuse it.

Get patterns

No pattern is included in the package. The hyph-utf8 project publishes patterns for many languages, each under its own license: use the plain text files (hyph-<language>.pat.txt and hyph-<language>.hyp.txt).

• Load and LoadAsync read UTF-8 files or streams; Parse reads text or UTF-8 bytes.
• The patterns are words with digits (hy3ph), separated by whitespace, optionally inside a \patterns{...} group; the exceptions are hyphenated words (as-so-ciate), in a separate file or in a \hyphenation{...} group. An exception without a hyphen is never broken, and % starts a comment.
• Only the plain text form is read. A file that uses TeX macros, \lccode or the ^^ notation is rejected with a FormatException.

Options

HyphenatorOptions is given when the patterns are loaded:

• The right values of LeftHyphenMin and RightHyphenMin depend on the language: they are the \lefthyphenmin and \righthyphenmin of TeX, and the files of hyph-utf8 give them. A character is a grapheme cluster.
• The limits protect you from pattern files from an untrusted source: a file that exceeds one is rejected with a FormatException before memory is allocated for it.

Hyphenation that changes the spelling of a word is not supported.

To hyphenate the paragraphs drawn on images, give the hyphenator to the text style of Meziantou.Framework.Imaging.