Parsing documents

HtmlDocumentParser implements the parsing algorithm of the HTML Standard: it builds the same tree as a browser for any input, valid or not. It never throws on invalid markup.

Parse a string

HtmlDocumentParser.ParseDocument(html) parses a document from a string:

var document = HtmlDocumentParser.ParseDocument("""
    <style>.important { color: red; font-weight: bold }</style>
    <ul>
      <li><a href="https://example.com/a" class="important">A</a>
      <li><a href="/b" style="color: blue">B</a>
    </ul>
    """);

// CSS selectors, with typed overloads and LINQ
var links = document.QuerySelectorAll<HtmlAnchorElement>("li > a");
var external = links.Where(a => a.Href.StartsWith("https:", StringComparison.Ordinal)).Select(a => a.TextContent);
var first = document.GetElementsByTagName("li")[0];
Assert.Equal(["A"], external);
Assert.Same(first, links[0].ParentElement);

// XPath
var result = document.Evaluate("//a[@class='important']", document, type: XPathResult.FirstOrderedNodeType);
var important = (Element?)result.SingleNodeValue;
Assert.Equal("A", important?.TextContent);

// Inline style (style attribute)
var color = links[1].Style.GetPropertyValue("color");
Assert.Equal("blue", color);

The missing elements are created as in a browser: the document always has an html element with a head and a body. A string has no encoding to determine, so the options about encodings do not apply.

Parse bytes and streams

A file or an HTTP response is a sequence of bytes. Parse the bytes rather than a string decoded beforehand: the parser finds the encoding as a browser does, from the byte order mark, the <meta charset> element, or the content itself.

// The bytes of a page in windows-1252, as they come from a file or an HTTP response
var bytes = Encoding.Latin1.GetBytes("<meta charset=windows-1252><p>Café</p>");

// The encoding is found from the bytes: byte order mark, <meta charset>, or detected from the content
var document = new HtmlDocumentParser().Parse(bytes);
Assert.Equal("windows-1252", document.CharacterSet);
Assert.Equal("Café", document.QuerySelector("p")!.TextContent);

// The charset of the Content-Type header wins over the document, and Url resolves the relative URLs
var parser = new HtmlDocumentParser(new HtmlParserOptions
{
    TransportEncoding = "utf-8",
    Url = "https://example.com/blog/",
});
using var stream = new MemoryStream("<a href='first-post'>Café</a>"u8.ToArray());
var post = await parser.ParseAsync(stream);
var link = post.QuerySelector<HtmlAnchorElement>("a")!;
Assert.Equal("https://example.com/blog/first-post", link.Href);
Assert.Equal("Café", link.TextContent);

HtmlDocumentParser instances have the options and the overloads Parse(string), Parse(ReadOnlySpan<byte>), Parse(Stream), ParseAsync(string) and ParseAsync(Stream). The HtmlParserOptions are:

Report the parse errors

The parse errors do not stop the parser: they describe what a browser fixed. Collect them to validate generated or hand-written HTML:

var errors = new List<HtmlParseError>();
var document = HtmlDocumentParser.ParseDocument("<p>Unclosed <b>bold</p>", new HtmlParserOptions
{
    ParseErrorHandler = errors.Add,
});

// The errors are reported, and the document is built as a browser builds it
Assert.Equal("<p>Unclosed <b>bold</b></p>", document.Body!.InnerHtml);
Assert.Equal(["(1,3): expected-doctype-but-got-other", "(1,23): unexpected-end-tag"], errors.Select(error => error.ToString()));
Assert.Equal((1, 3), (errors[0].Line, errors[0].Column));

Each HtmlParseError has the Code of the HTML Standard, the Line and the Column, and the Offset of the character in the input (in UTF-16 code units). Tracking the positions costs nothing when there is no handler.

Parse a fragment

A fragment is parsed in the context of an element, because the same markup gives different nodes in different places: <td> creates cells inside a table row, and is ignored inside a div.

var document = HtmlDocumentParser.ParseDocument("<table><tr id=row></tr></table>");
var row = document.GetElementById("row")!;

// The markup is parsed as the content of the context element: here, cells of a table row
var fragment = new HtmlDocumentParser().ParseFragment("<td>a<td>b", row);
row.Append(fragment);
Assert.Equal("<td>a</td><td>b</td>", row.InnerHtml);

// The same, in one step
row.InnerHtml = "<td>c<td>d";
row.InsertAdjacentHtml("beforeend", "<td>e");
Assert.Equal("<td>c</td><td>d</td><td>e</td>", row.InnerHtml);

InnerHtml, OuterHtml and InsertAdjacentHtml parse the markup in the context of the element. To insert markup that comes from users, use SetHtml instead: see Sanitizing HTML.

Parse XML

XmlDocumentParser.ParseDocument parses an XML document: XHTML, SVG, feeds or any other XML vocabulary. The elements are in their namespaces, and the XHTML elements get their typed interfaces.

var feed = XmlDocumentParser.ParseDocument("""
    <?xml version="1.0" encoding="utf-8"?>
    <feed xmlns="http://www.w3.org/2005/Atom">
      <entry><title>First post</title></entry>
      <entry><title>Second post</title></entry>
    </feed>
    """);

var titles = feed.GetElementsByTagNameNS("http://www.w3.org/2005/Atom", "title").Select(title => title.TextContent);
Assert.Equal(["First post", "Second post"], titles);

// A document that is not well-formed throws
Assert.Throws<XmlException>(() => XmlDocumentParser.ParseDocument("<root><item></root>"));

// DOMParser returns a parsererror document instead, as in a browser
var invalid = new DomParser().ParseFromString("<root><item></root>", DomParserSupportedType.ApplicationXml);
Assert.Equal("parsererror", invalid.DocumentElement!.LocalName);

• Parse XML bytes with ParseDocument(ReadOnlySpan<byte>), or streams with ParseDocument(Stream) and ParseDocumentAsync. The byte order mark and XML byte signature determine the byte order; the XML declaration selects a supported .NET encoding. Without either, the encoding is UTF-8. Malformed byte sequences, unsupported encodings and declarations conflicting with a byte order mark or signature throw XmlException. Strings are already decoded: their declarations never change their characters.
• Stream overloads read from the current position, buffer the input and leave the caller-owned stream open, including on errors or cancellation. The asynchronous overload honors cancellation during reading and while waiting for the parser; DOM construction itself is synchronous. These overloads do not promise incremental or constant-memory parsing.
• External entities are never fetched, and the entity expansion limit also applies to byte and stream inputs.

var bytes = Encoding.Latin1.GetBytes("<?xml version='1.0' encoding='iso-8859-1'?><root>caf\u00e9</root>");
var document = XmlDocumentParser.ParseDocument(bytes);
Assert.Equal("caf\u00e9", document.DocumentElement!.TextContent);

using var stream = new MemoryStream(bytes);
var streamed = await XmlDocumentParser.ParseDocumentAsync(stream, cancellationToken: CancellationToken.None);
Assert.Equal("caf\u00e9", streamed.DocumentElement!.TextContent);
Assert.True(stream.CanRead);

• XmlParserOptions.ContentType is the type of the document: application/xml (the default), text/xml, application/xhtml+xml or image/svg+xml. Url is its URL.
• Use GetElementsByTagNameNS and the namespace-aware members (NamespaceUri, LocalName) with XML: element names are case-sensitive.
• XmlSerializer writes a node back as XML (new XmlSerializer().SerializeToString(node)).

Documents and browsing contexts

A parsed document has no browsing context: its scripts never run, and there is no window, computed style or layout. To load a document as a page, navigate a browsing context to it; the browsing context uses the same parser, with the URL and the encoding of the response. See Loading pages.