Meziantou.FrameworkEncodingDetector Class
Detects the character encoding of the HTML documents that do not declare one: the step of the encoding sniffing algorithm that "may attempt to autodetect the character encoding from applying frequency analysis or other algorithms to the data stream" (https://html.spec.whatwg.org/multipage/parsing.html#determining-the-character-encoding). It runs after the byte order mark, the encoding of the transport layer, the meta elements and the encoding of the container document, and before the default encoding; the confidence of its result is tentative.
Set HtmlParserOptions.EncodingDetector or BrowsingContextOptions.EncodingDetector to replace the detector (EncodingDetector.Default), or to null to disable the detection.
public abstract class EncodingDetectorConstructors
| Name | Description |
|---|---|
| EncodingDetector() |
EncodingDetector()
protected EncodingDetector()No documentation is available.
Properties
| Name | Description |
|---|---|
| Default | Gets the built-in detector, a port of chardetng, the detector of Firefox: it guesses the legacy encodings from the validity of the byte sequences and the frequencies of the characters of the languages that use them (Shift_JIS, EUC-JP, GBK, Big5, EUC-KR, windows-1250 to windows-1258, windows-874, ISO-8859-2, ISO-8859-4 to ISO-8859-8, ISO-8859-13, KOI8-U and IBM866). As in the browsers, UTF-8 is only detected for the documents that do not come from the network: the ones whose URL is a file: URL, and the ones that have no URL (HtmlParserOptions.Url is not set). The other documents whose bytes are valid UTF-8 get the default encoding, so that the content of the Web does not start to rely on the detection instead of declaring its encoding. ISO-2022-JP is never detected. The validity of UTF-8 is checked for all the bytes; the legacy encodings are guessed from the 256 KiB that follow the first non-ASCII byte. |
Default
EncodingDetector.csGets the built-in detector, a port of chardetng, the detector of Firefox: it guesses the legacy encodings from the validity of the byte sequences and the frequencies of the characters of the languages that use them (Shift_JIS, EUC-JP, GBK, Big5, EUC-KR, windows-1250 to windows-1258, windows-874, ISO-8859-2, ISO-8859-4 to ISO-8859-8, ISO-8859-13, KOI8-U and IBM866).
As in the browsers, UTF-8 is only detected for the documents that do not come from the network: the ones whose URL is a file: URL, and the ones that have no URL (HtmlParserOptions.Url is not set). The other documents whose bytes are valid UTF-8 get the default encoding, so that the content of the Web does not start to rely on the detection instead of declaring its encoding. ISO-2022-JP is never detected.
The validity of UTF-8 is checked for all the bytes; the legacy encodings are guessed from the 256 KiB that follow the first non-ASCII byte.
public static EncodingDetector Default { get; }Property value
Methods
| Name | Description |
|---|---|
| Detect(ReadOnlySpan<byte>, EncodingDetectionContext) | Detects the encoding of a document. It is not called for the documents that only have ASCII bytes other than the escape, shift out and shift in characters (0x1B, 0x0E and 0x0F): they are the same in all the encodings that can be detected. |
Detect(ReadOnlySpan<byte>, EncodingDetectionContext)
Detects the encoding of a document. It is not called for the documents that only have ASCII bytes other than the escape, shift out and shift in characters (0x1B, 0x0E and 0x0F): they are the same in all the encodings that can be detected.
public abstract string? Detect(ReadOnlySpan<byte> bytes, EncodingDetectionContext context)Parameters
bytesReadOnlySpan<byte>- The bytes of the document: all of them, or, for a document that is parsed as its bytes arrive, the bytes received when the first chunk that has another byte than the ones above arrives (see
EncodingDetectionContext.IsComplete). contextEncodingDetectionContext- The context of the detection.
Returns
string? A label of the encoding (https://encoding.spec.whatwg.org/#names-and-labels, e.g. windows-1251), or null when the encoding is not detected: the document gets the default encoding. The labels that are not supported and the ones of UTF-16BE/LE and of the replacement encoding are ignored.