Meziantou.Framework.RobotsTxt namespace

Classes

Name	Description
RobotsFile	Represents a parsed `robots.txt` file.
RobotsGroup	Represents a group of directives in a `robots.txt` file that applies to one or more user-agents.
RobotsParseError	Describes a single non-fatal parse error encountered while reading a `robots.txt` file.
RobotsRule	Represents a single `Allow` or `Disallow` directive in a `robots.txt` file.

Name	Description
RobotsParseErrorKind	Identifies the kind of parse error encountered while reading a `robots.txt` file.
RobotsRuleKind	Specifies whether a `RobotsRule` allows or disallows access to a path.