Hyphenator Class

Namespace: Meziantou.Framework
Assembly: Meziantou.Framework.Unicode.dll
Package: Meziantou.Framework.Unicode 2.0.15
Source: Hyphenator.cs
Available in package assets: net10.0net11.0

Finds the positions where a word can be hyphenated, from the hyphenation patterns of a language.

public sealed class Hyphenator

Remarks

The Unicode Standard defines where a line can break (see Unicode.EnumerateLineBreaks), including after a soft hyphen, but hyphenating a word needs data for its language. This class applies the pattern algorithm of Franklin Liang, the one of TeX, to patterns that the caller provides: no pattern is included in this package. Patterns for many languages are published by the hyph-utf8 project of the TeX community, each under its own license.

The accepted input is the plain text form of the patterns:

Patterns separated by white space, such as hy3ph he2n .un1: a digit between two letters gives the value of a break at that position, and a dot stands for the start or the end of a word.
Optionally inside a \patterns{...} group, with the exceptions in a \hyphenation{...} group, as in a TeX file.
Exceptions are words with their breaks, such as as-so-ciate; a word without a hyphen is never broken.
% starts a comment that ends with the line.

No other TeX command is interpreted: a file that uses macros, \lccode or the ^^ notation is rejected. Hyphenation that changes the spelling of a word is not supported.

An instance is immutable and can be used by several threads at the same time.

Properties

Name	Description
ExceptionCount	Gets the number of exceptions.
LeftHyphenMin	Gets the smallest number of characters before the first break of a word.
PatternCount	Gets the number of distinct patterns.
RightHyphenMin	Gets the smallest number of characters after the last break of a word.

ExceptionCount

Hyphenator.cs

Gets the number of exceptions.

public int ExceptionCount { get; }

Property value

int

LeftHyphenMin

Hyphenator.cs

Gets the smallest number of characters before the first break of a word.

public int LeftHyphenMin { get; }

Property value

int

PatternCount

Hyphenator.cs

Gets the number of distinct patterns.

public int PatternCount { get; }

Property value

int

RightHyphenMin

Hyphenator.cs

Gets the smallest number of characters after the last break of a word.

public int RightHyphenMin { get; }

Property value

int

Methods

Name	Description
Hyphenate(ReadOnlySpan<char>, Span<int>)	Finds the positions where a word can be hyphenated.
Load(Stream, Stream?, HyphenatorOptions?)	Loads the patterns of a language from UTF-8 streams.
Load(string, string?, HyphenatorOptions?)	Loads the patterns of a language from UTF-8 files.
LoadAsync(Stream, Stream?, HyphenatorOptions?, CancellationToken)	Loads the patterns of a language from UTF-8 streams.
LoadAsync(string, string?, HyphenatorOptions?, CancellationToken)	Loads the patterns of a language from UTF-8 files.
Parse(ReadOnlySpan<byte>, ReadOnlySpan<byte>, HyphenatorOptions?)	Creates a hyphenator from the UTF-8 text of the patterns of a language.
Parse(ReadOnlySpan<char>, ReadOnlySpan<char>, HyphenatorOptions?)	Creates a hyphenator from the text of the patterns of a language.

Hyphenate(ReadOnlySpan<char>, Span<int>)

Hyphenator.cs

Finds the positions where a word can be hyphenated.

public int Hyphenate(ReadOnlySpan<char> word, Span<int> breakPositions)

Parameters

word ReadOnlySpan<char>: The word, without the punctuation or the spaces around it. Its case does not matter.
breakPositions Span<int>: The buffer that receives the positions, in increasing order. A position is the index of the UTF-16 code unit of word before which a hyphen can be inserted. As many elements as the length of the word are always enough.

Returns

int The number of positions written; 0 when the word cannot be hyphenated, or is longer than HyphenatorOptions.MaxWordLength.

Exceptions

ArgumentException: breakPositions is too small.

Remarks

A break is never inside a grapheme cluster, and it leaves at least Hyphenator.LeftHyphenMin characters before it and Hyphenator.RightHyphenMin after it. The method does not allocate.

Load overloads

Load(Stream, Stream?, HyphenatorOptions?)

Hyphenator.cs

Loads the patterns of a language from UTF-8 streams.

public static Hyphenator Load(Stream patterns, Stream? exceptions = null, HyphenatorOptions? options = null)

Parameters

patterns Stream: The stream of the patterns. It can also contain the exceptions, in a \hyphenation{...} group. It is read to its end and left open.
exceptions Stream?: The stream of the exceptions, or null. It is read to its end and left open.
options HyphenatorOptions?: The options, or null for HyphenatorOptions.Default.

Returns

Hyphenator The hyphenator.

Exceptions

ArgumentNullException: patterns is null.
FormatException: The content is not valid UTF-8, is malformed, or exceeds a limit of options.

Load(string, string?, HyphenatorOptions?)

Hyphenator.cs

Loads the patterns of a language from UTF-8 files.

public static Hyphenator Load(string patternsPath, string? exceptionsPath = null, HyphenatorOptions? options = null)

Parameters

patternsPath string: The path of the file of the patterns. It can also contain the exceptions, in a \hyphenation{...} group.
exceptionsPath string?: The path of the file of the exceptions, or null.
options HyphenatorOptions?: The options, or null for HyphenatorOptions.Default.

Returns

Hyphenator The hyphenator.

Exceptions

ArgumentNullException: patternsPath is null.
FormatException: The content is not valid UTF-8, is malformed, or exceeds a limit of options.
IOException: A file cannot be read.

LoadAsync overloads

LoadAsync(Stream, Stream?, HyphenatorOptions?, CancellationToken)

Hyphenator.cs

Loads the patterns of a language from UTF-8 streams.

net10.0

public static Task<Hyphenator> LoadAsync(Stream patterns, Stream? exceptions = null, HyphenatorOptions? options = null, CancellationToken cancellationToken = default)

net11.0

public static Task<Hyphenator> LoadAsync(Stream patterns, Stream? exceptions = null, HyphenatorOptions? options = null, CancellationToken cancellationToken = default)

Parameters

patterns Stream: The stream of the patterns. It can also contain the exceptions, in a \hyphenation{...} group. It is read to its end and left open.
exceptions Stream?: The stream of the exceptions, or null. It is read to its end and left open.
options HyphenatorOptions?: The options, or null for HyphenatorOptions.Default.
cancellationToken CancellationToken: The token to monitor for cancellation requests.

Returns

Task<Hyphenator> The hyphenator.

Exceptions

ArgumentNullException: patterns is null.
FormatException: The content is not valid UTF-8, is malformed, or exceeds a limit of options.
OperationCanceledException: The operation was canceled.

LoadAsync(string, string?, HyphenatorOptions?, CancellationToken)

Hyphenator.cs

Loads the patterns of a language from UTF-8 files.

net10.0

public static Task<Hyphenator> LoadAsync(string patternsPath, string? exceptionsPath = null, HyphenatorOptions? options = null, CancellationToken cancellationToken = default)

net11.0

public static Task<Hyphenator> LoadAsync(string patternsPath, string? exceptionsPath = null, HyphenatorOptions? options = null, CancellationToken cancellationToken = default)

Parameters

patternsPath string: The path of the file of the patterns. It can also contain the exceptions, in a \hyphenation{...} group.
exceptionsPath string?: The path of the file of the exceptions, or null.
options HyphenatorOptions?: The options, or null for HyphenatorOptions.Default.
cancellationToken CancellationToken: The token to monitor for cancellation requests.

Returns

Task<Hyphenator> The hyphenator.

Exceptions

ArgumentNullException: patternsPath is null.
FormatException: The content is not valid UTF-8, is malformed, or exceeds a limit of options.
IOException: A file cannot be read.
OperationCanceledException: The operation was canceled.

Parse overloads

Parse(ReadOnlySpan<byte>, ReadOnlySpan<byte>, HyphenatorOptions?)

Hyphenator.cs

Creates a hyphenator from the UTF-8 text of the patterns of a language.

public static Hyphenator Parse(ReadOnlySpan<byte> utf8Patterns, ReadOnlySpan<byte> utf8Exceptions = default, HyphenatorOptions? options = null)

Parameters

utf8Patterns ReadOnlySpan<byte>: The patterns, in UTF-8. They can also contain the exceptions, in a \hyphenation{...} group.
utf8Exceptions ReadOnlySpan<byte>: The exceptions, in UTF-8, or an empty span.
options HyphenatorOptions?: The options, or null for HyphenatorOptions.Default.

Returns

Hyphenator The hyphenator.

Exceptions

FormatException: The content is not valid UTF-8, is malformed, or exceeds a limit of options.

Parse(ReadOnlySpan<char>, ReadOnlySpan<char>, HyphenatorOptions?)

Hyphenator.cs

Creates a hyphenator from the text of the patterns of a language.

public static Hyphenator Parse(ReadOnlySpan<char> patterns, ReadOnlySpan<char> exceptions = default, HyphenatorOptions? options = null)

Parameters

patterns ReadOnlySpan<char>: The patterns. They can also contain the exceptions, in a \hyphenation{...} group.
exceptions ReadOnlySpan<char>: The exceptions, or an empty span.
options HyphenatorOptions?: The options, or null for HyphenatorOptions.Default.

Returns

Hyphenator The hyphenator.

Exceptions

FormatException: The content is malformed, or exceeds a limit of options.