Click here to Skip to main content
13,351,954 members (80,462 online)
Click here to Skip to main content
Add your own
alternative version


26 bookmarked
Posted 11 Jun 2009

AfterWork HTML Parser in C#

, 11 Jun 2009
Rate this:
Please Sign up or sign in to vote.
Actually, this is more of a lexical analyzer, but still very applicable for reading HTML and building a DOM tree.


This is a tribute to MIL HTML Parser which I used couple of times and which turned out to be not capable of reading some HTMLs around.


This is an HTML lexical analyzer, which is one step away from a decent HTML parser. Basically, the only difference is that this analyzer produces a sequence of HTML tokens and doesn't build an HTML tree-structure.

This thing is well-trained to handle many situations of reading loosely formatted HTML pages (which are pretty common in the Internet).

Using the code

Here are a couple of examples to get a quick introduction of how this thing works.

In the following example, we take a page from eBay and see the sequence of HTML tokens produced by the lexer:

public void DemoExampleTest()
    HttpWebRequest request = (HttpWebRequest)WebRequest.Create("");
    HttpWebResponse response = (HttpWebResponse)request.GetResponse();
    string html;
    using (StreamReader streamReader = new StreamReader(response.GetResponseStream()))
        html = streamReader.ReadToEnd();
    HtmlReader reader = new HtmlReader(html);
    IndentBuilder tracker = new IndentBuilder();
    HtmlReader.Read(reader, tracker);
    // Trace.WriteLine(tracker.ToString());  // << == UNCOMMENT WITH CAUTION!!!

/* -- This is what you are likely to see  ---
[WHITESPACE: ][DTD_FPI:"-//W3C//DTD HTML 4.01 Transitional//EN"][WHITESPACE: ]
[ATTR:http-equiv][ASSIGN:=][QUOTED_VALUE:"Content-Type"][WHITESPACE: ][ATTR:content]
[ASSIGN:=][QUOTED_VALUE:"text/html; charset=UTF-8"][TAG_ENDS:>][TAG_STARTS:<][NAME:link] ...

In the next example, we work with the same page with some adjustments to the HTML grammar being used. Take a look at the handler of the TokenChaning event, this is a very good place to put your code for building an HTML tree-structure.

public void PracticalExampleTest()
    HttpWebRequest request = (HttpWebRequest)WebRequest.Create("");
    HttpWebResponse response = (HttpWebResponse)request.GetResponse();
    string html;
    using (StreamReader streamReader = new StreamReader(response.GetResponseStream()))
        html = streamReader.ReadToEnd();
    HtmlGrammarOptions options = new HtmlGrammarOptions();
    options.HandleCharacterReferences = true;
    options.DecomposeCharacterReference = true;
    options.HandleUnfinishedTags = true;
    HtmlGrammar grammar = new HtmlGrammar(options);
    HtmlReader reader = new HtmlReader(html, grammar);
    reader.Builder.TokenChaning += delegate(TokenChangingArgs args)
        if (args.HasBefore)
            Trace.WriteLine(args.Before.Id + ": " + args.Before.Value);
    HtmlReader.Read(reader, null);

Points of interest

I personally believe that there must be a free decent HTML parser in the public domain. It took me several days to realize that there is nothing in the Internet that can be used out of the box in a C# .NET project for this purpose. Well, let's try to make a difference.


This is actually a prototype of a parser I've been working on lately. Which means this thing needs some additional work to be finished. But it's somewhat already functional, and reads 98% of HTMLs you would ever see. There are a few cases where it can suddenly stop, but just have a quick look at the HtmlGrammar class. All missing links and states can be easily added there.

See also

There are also some useful stuff you might be interested in:


This article, along with any associated source code and files, is licensed under A Public Domain dedication


About the Author

Aleksey Bykov
United States United States
C# .NET developer since 2003

You may also be interested in...

Comments and Discussions

Question不能下载,网页打开非常缓慢 Pin
Member 126762848-Aug-16 21:03
memberMember 126762848-Aug-16 21:03 
QuestionHow to run this program Pin
shiblybcc16-Sep-12 23:51
membershiblybcc16-Sep-12 23:51 
AnswerRe: How to run this program Pin
privacy-concerned-guy17-Sep-12 2:55
memberprivacy-concerned-guy17-Sep-12 2:55 
GeneralHTML Agilty kit Pin
Ismail Mayat11-Jun-09 6:57
memberIsmail Mayat11-Jun-09 6:57 
QuestionDid you look at SGMLReader? Pin
HightechRider11-Jun-09 6:43
memberHightechRider11-Jun-09 6:43 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

Permalink | Advertise | Privacy | Terms of Use | Mobile
Web04 | 2.8.180111.1 | Last Updated 11 Jun 2009
Article Copyright 2009 by Aleksey Bykov
Everything else Copyright © CodeProject, 1999-2018
Layout: fixed | fluid