Convert HTML to Plain Text

Introduction

This article provides the procedure for stripping out HTML tags while preserving most basic formatting. In other words, it converts HTML to plain text.

Background

This example heavily relies on regular expressions, in particular System.Text.RegularExpressions.Regex.Replace() method. You may also find this reference on regular expressions syntax useful.

Using the Code

The code uses System.Text.RegularExpressions namespace and consists of a single function, StripHTML().

First, the development formatting is removed such as tabs used for step-identations and repeated whitespaces. As a result, the input HTML is "flattened" into one continuous string. This serves two reasons:

To remove the formatting ignored by browsers
To make the regexes work reliably (they seem to get confused by escape characters)

Then the header is removed by removing anything between <head> and </head> tags.

Then, all scripts are removed by chopping out anything between <script> and </script> tags inclusive. Similarly with styles.

Then the basic formatting tags, such as <BR> and <DIV> are replaced with \r or \r\r. Also <TR> tags are replaced by line breaks and <TD>s by tabs.

<LI>s are replaced by *s and special characters such as are replaced with their corresponding values.

Finally all the remaining tags are replaced with empty strings.

By this stage, there are likely to be a lot of redundant repeating line breaks and tabs. Any sequence over 2 line breaks long is replaced by two line breaks. Similarly with tabs: sequences over 4 tabs long are replaced by 4 tabs.

private string StripHTML(string source)
{
    try
    {
        string result;

        // Remove HTML Development formatting
        // Replace line breaks with space
        // because browsers inserts space
        result = source.Replace("\r", " ");
        // Replace line breaks with space
        // because browsers inserts space
        result = result.Replace("\n", " ");
        // Remove step-formatting
        result = result.Replace("\t", string.Empty);
        // Remove repeating spaces because browsers ignore them
        result = System.Text.RegularExpressions.Regex.Replace(result,
                                                              @"( )+", " ");

        // Remove the header (prepare first by clearing attributes)
        result = System.Text.RegularExpressions.Regex.Replace(result,
                 @"<( )*head([^>])*>","<head>",
                 System.Text.RegularExpressions.RegexOptions.IgnoreCase);
        result = System.Text.RegularExpressions.Regex.Replace(result,
                 @"(<( )*(/)( )*head( )*>)","</head>",
                 System.Text.RegularExpressions.RegexOptions.IgnoreCase);
        result = System.Text.RegularExpressions.Regex.Replace(result,
                 "(<head>).*(</head>)",string.Empty,
                 System.Text.RegularExpressions.RegexOptions.IgnoreCase);

        // remove all scripts (prepare first by clearing attributes)
        result = System.Text.RegularExpressions.Regex.Replace(result,
                 @"<( )*script([^>])*>","<script>",
                 System.Text.RegularExpressions.RegexOptions.IgnoreCase);
        result = System.Text.RegularExpressions.Regex.Replace(result,
                 @"(<( )*(/)( )*script( )*>)","</script>",
                 System.Text.RegularExpressions.RegexOptions.IgnoreCase);
        //result = System.Text.RegularExpressions.Regex.Replace(result,
        //         @"(<script>)([^(<script>\.</script>)])*(</script>)",
        //         string.Empty,
        //         System.Text.RegularExpressions.RegexOptions.IgnoreCase);
        result = System.Text.RegularExpressions.Regex.Replace(result,
                 @"(<script>).*(</script>)",string.Empty,
                 System.Text.RegularExpressions.RegexOptions.IgnoreCase);

        // remove all styles (prepare first by clearing attributes)
        result = System.Text.RegularExpressions.Regex.Replace(result,
                 @"<( )*style([^>])*>","<style>",
                 System.Text.RegularExpressions.RegexOptions.IgnoreCase);
        result = System.Text.RegularExpressions.Regex.Replace(result,
                 @"(<( )*(/)( )*style( )*>)","</style>",
                 System.Text.RegularExpressions.RegexOptions.IgnoreCase);
        result = System.Text.RegularExpressions.Regex.Replace(result,
                 "(<style>).*(</style>)",string.Empty,
                 System.Text.RegularExpressions.RegexOptions.IgnoreCase);

        // insert tabs in spaces of <td> tags
        result = System.Text.RegularExpressions.Regex.Replace(result,
                 @"<( )*td([^>])*>","\t",
                 System.Text.RegularExpressions.RegexOptions.IgnoreCase);

        // insert line breaks in places of <BR> and <LI> tags
        result = System.Text.RegularExpressions.Regex.Replace(result,
                 @"<( )*br( )*>","\r",
                 System.Text.RegularExpressions.RegexOptions.IgnoreCase);
        result = System.Text.RegularExpressions.Regex.Replace(result,
                 @"<( )*li( )*>","\r",
                 System.Text.RegularExpressions.RegexOptions.IgnoreCase);

        // insert line paragraphs (double line breaks) in place
        // if <P>, <DIV> and <TR> tags
        result = System.Text.RegularExpressions.Regex.Replace(result,
                 @"<( )*div([^>])*>","\r\r",
                 System.Text.RegularExpressions.RegexOptions.IgnoreCase);
        result = System.Text.RegularExpressions.Regex.Replace(result,
                 @"<( )*tr([^>])*>","\r\r",
                 System.Text.RegularExpressions.RegexOptions.IgnoreCase);
        result = System.Text.RegularExpressions.Regex.Replace(result,
                 @"<( )*p([^>])*>","\r\r",
                 System.Text.RegularExpressions.RegexOptions.IgnoreCase);

        // Remove remaining tags like <a>, links, images,
        // comments etc - anything that's enclosed inside < >
        result = System.Text.RegularExpressions.Regex.Replace(result,
                 @"<[^>]*>",string.Empty,
                 System.Text.RegularExpressions.RegexOptions.IgnoreCase);

        // replace special characters:
        result = System.Text.RegularExpressions.Regex.Replace(result,
                 @" "," ",
                 System.Text.RegularExpressions.RegexOptions.IgnoreCase);

        result = System.Text.RegularExpressions.Regex.Replace(result,
                 @"&bull;"," * ",
                 System.Text.RegularExpressions.RegexOptions.IgnoreCase);
        result = System.Text.RegularExpressions.Regex.Replace(result,
                 @"&lsaquo;","<",
                 System.Text.RegularExpressions.RegexOptions.IgnoreCase);
        result = System.Text.RegularExpressions.Regex.Replace(result,
                 @"&rsaquo;",">",
                 System.Text.RegularExpressions.RegexOptions.IgnoreCase);
        result = System.Text.RegularExpressions.Regex.Replace(result,
                 @"&trade;","(tm)",
                 System.Text.RegularExpressions.RegexOptions.IgnoreCase);
        result = System.Text.RegularExpressions.Regex.Replace(result,
                 @"&frasl;","/",
                 System.Text.RegularExpressions.RegexOptions.IgnoreCase);
        result = System.Text.RegularExpressions.Regex.Replace(result,
                 @"&lt;","<",
                 System.Text.RegularExpressions.RegexOptions.IgnoreCase);
        result = System.Text.RegularExpressions.Regex.Replace(result,
                 @"&gt;",">",
                 System.Text.RegularExpressions.RegexOptions.IgnoreCase);
        result = System.Text.RegularExpressions.Regex.Replace(result,
                 @"&copy;","(c)",
                 System.Text.RegularExpressions.RegexOptions.IgnoreCase);
        result = System.Text.RegularExpressions.Regex.Replace(result,
                 @"&reg;","(r)",
                 System.Text.RegularExpressions.RegexOptions.IgnoreCase);
        // Remove all others. More can be added, see
        // http://hotwired.lycos.com/webmonkey/reference/special_characters/
        result = System.Text.RegularExpressions.Regex.Replace(result,
                 @"&(.{2,6});", string.Empty,
                 System.Text.RegularExpressions.RegexOptions.IgnoreCase);

        // for testing
        //System.Text.RegularExpressions.Regex.Replace(result,
        //       this.txtRegex.Text,string.Empty,
        //       System.Text.RegularExpressions.RegexOptions.IgnoreCase);

        // make line breaking consistent
        result = result.Replace("\n", "\r");

        // Remove extra line breaks and tabs:
        // replace over 2 breaks with 2 and over 4 tabs with 4.
        // Prepare first to remove any whitespaces in between
        // the escaped characters and remove redundant tabs in between line breaks
        result = System.Text.RegularExpressions.Regex.Replace(result,
                 "(\r)( )+(\r)","\r\r",
                 System.Text.RegularExpressions.RegexOptions.IgnoreCase);
        result = System.Text.RegularExpressions.Regex.Replace(result,
                 "(\t)( )+(\t)","\t\t",
                 System.Text.RegularExpressions.RegexOptions.IgnoreCase);
        result = System.Text.RegularExpressions.Regex.Replace(result,
                 "(\t)( )+(\r)","\t\r",
                 System.Text.RegularExpressions.RegexOptions.IgnoreCase);
        result = System.Text.RegularExpressions.Regex.Replace(result,
                 "(\r)( )+(\t)","\r\t",
                 System.Text.RegularExpressions.RegexOptions.IgnoreCase);
        // Remove redundant tabs
        result = System.Text.RegularExpressions.Regex.Replace(result,
                 "(\r)(\t)+(\r)","\r\r",
                 System.Text.RegularExpressions.RegexOptions.IgnoreCase);
        // Remove multiple tabs following a line break with just one tab
        result = System.Text.RegularExpressions.Regex.Replace(result,
                 "(\r)(\t)+","\r\t",
                 System.Text.RegularExpressions.RegexOptions.IgnoreCase);
        // Initial replacement target string for line breaks
        string breaks = "\r\r\r";
        // Initial replacement target string for tabs
        string tabs = "\t\t\t\t\t";
        for (int index=0; index<result.Length; index++)
        {
            result = result.Replace(breaks, "\r\r");
            result = result.Replace(tabs, "\t\t\t\t");
            breaks = breaks + "\r";
            tabs = tabs + "\t";
        }

        // That's it.
        return result;
    }
    catch
    {
        MessageBox.Show("Error");
        return source;
    }
}

Points of Interest

Escape characters such as \n and \r had to be removed first because they cause regexes to cease working as expected.

Moreover, to make the result string display correctly in the textbox, one might need to split it up and set textbox's Lines property instead of assigning to Text property.

this.txtResult.Lines =
      StripHTML(this.txtSource.Text).Split("\r".ToCharArray());

History

6^th October, 2005: Initial post

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

About the Author

paceman

Web Developer

Australia

Member

Article Top

Poor

Excellent

Add a reason or comment to your vote: x
Votes of 3 or less require a comment

Comments and Discussions

You must Sign In to use this message board. (secure sign-in)

Search this forum
	Profile popups Noise Layout Per page

What about acute characters? [modified]

Mihaly Sogorka

4:44 6 Apr '12

Like: "& O a c u t e;" -> Ó This characters are trunclated during the process... I think this is a bug. modified 6 Apr '12.
Sign In·View Thread·Permalink

error in the code

deepika deepi

2:13 3 Apr '12

hello sir, i am getting error in the html to text converter code. as illegal start up expression @"( )+", " "); like this many errors. can you kindly tell what wrong with the coding level i have done? and send the full code with the file name till the end to my mail id ?
Sign In·View Thread·Permalink

html to text

parvez12

6:48 2 Dec '11

I used below code, it is working fine for me static String hTML2Text(String text) { String newText = text.trim(); newText = newText.trim().replaceAll(" ", "").replaceAll("", "").replaceAll("\n", "").replaceAll("\r", "").replaceAll("\t", "").trim(); return newText; } Parvez Ahmad Hakim www.abobjects.com
Sign In·View Thread·Permalink	1.00/5 (1 vote)

hmm

vlad781

19:51 7 Jun '11

ok, lets say i have a web browser... there is a number named 11. however its value is NOT 11, well, it can be anything. I want to take that value. how to i take that number value, and place that number into a label? MySignature.BringToFront();
Sign In·View Thread·Permalink	1.00/5 (3 votes)

Great Job!

SUFZOLI

18:06 8 Dec '10

Hi,

It is a great job what you have done here. I've rewriten the code a litlebit it may usefull

    public static class TextConverter
    {
        public static string ConvertHtmlToText(string source)
        {
            string AmpStr;
            MatchCollection AmpCodes;
 
            // Add all possible special character into a dictionary
            Dictionary<string, int> SpecChars = new Dictionary<string, int>();
            SpecChars.Add("&Aacute;", 193);     SpecChars.Add("&aacute;", 225);     SpecChars.Add("&Acirc;", 194);
            SpecChars.Add("&acirc;", 226);      SpecChars.Add("&acute;", 180);      SpecChars.Add("&AElig;", 198);
            SpecChars.Add("&aelig;", 230);      SpecChars.Add("&Agrave;", 192);     SpecChars.Add("&agrave;", 224);
            SpecChars.Add("&alefsym;", 8501);   SpecChars.Add("&Alpha;", 913);      SpecChars.Add("&alpha;", 945);
            SpecChars.Add("&amp;", 38);         SpecChars.Add("&and;", 8743);       SpecChars.Add("&ang;", 8736);
            SpecChars.Add("&Aring;", 197);      SpecChars.Add("&aring;", 229);      SpecChars.Add("&asymp;", 8776);
            SpecChars.Add("&Atilde;", 195);     SpecChars.Add("&atilde;", 227);     SpecChars.Add("&Auml;", 196);
            SpecChars.Add("&auml;", 228);       SpecChars.Add("&bdquo;", 8222);     SpecChars.Add("&Beta;", 914);
            SpecChars.Add("&beta;", 946);       SpecChars.Add("&brvbar;", 166);     SpecChars.Add("&bull;", 8226);
            SpecChars.Add("&cap;", 8745);       SpecChars.Add("&Ccedil;", 199);     SpecChars.Add("&ccedil;", 231);
            SpecChars.Add("&cedil;", 184);      SpecChars.Add("&cent;", 162);       SpecChars.Add("&Chi;", 935);
            SpecChars.Add("&chi;", 967);        SpecChars.Add("&circ;", 710);       SpecChars.Add("&clubs;", 9827);
            SpecChars.Add("&cong;", 8773);      SpecChars.Add("&copy;", 169);       SpecChars.Add("&crarr;", 8629);
            SpecChars.Add("&cup;", 8746);       SpecChars.Add("&curren;", 164);     SpecChars.Add("&dagger;", 8224);
            SpecChars.Add("&Dagger;", 8225);    SpecChars.Add("&darr;", 8595);      SpecChars.Add("&dArr;", 8659);
            SpecChars.Add("&deg;", 176);        SpecChars.Add("&Delta;", 916);      SpecChars.Add("&delta;", 948);
            SpecChars.Add("&diams;", 9830);     SpecChars.Add("&divide;", 247);     SpecChars.Add("&Eacute;", 201);
            SpecChars.Add("&eacute;", 233);     SpecChars.Add("&Ecirc;", 202);      SpecChars.Add("&ecirc;", 234);
            SpecChars.Add("&Egrave;", 200);     SpecChars.Add("&egrave;", 232);     SpecChars.Add("&emdash;", 8212);
            SpecChars.Add("&empty;", 8709);     SpecChars.Add("&emsp;", 8195);      SpecChars.Add("&endash;", 8211);
            SpecChars.Add("&ensp;", 8194);      SpecChars.Add("&Epsilon;", 917);    SpecChars.Add("&epsilon;", 949);
            SpecChars.Add("&equiv;", 8801);     SpecChars.Add("&Eta;", 919);        SpecChars.Add("&eta;", 951);
            SpecChars.Add("&ETH;", 208);        SpecChars.Add("&eth;", 240);        SpecChars.Add("&Euml;", 203);
            SpecChars.Add("&euml;", 235);       SpecChars.Add("&euro;", 8364);      SpecChars.Add("&exist;", 8707);
            SpecChars.Add("&fnof;", 402);       SpecChars.Add("&forall;", 8704);    SpecChars.Add("&frac12;", 189);
            SpecChars.Add("&frac14;", 188);     SpecChars.Add("&frac34;", 190);     SpecChars.Add("&frasl;", 8260);
            SpecChars.Add("&Gamma;", 915);      SpecChars.Add("&gamma;", 947);      SpecChars.Add("&ge;", 8805);
            SpecChars.Add("&gt;", 62);          SpecChars.Add("&harr;", 8596);      SpecChars.Add("&hArr;", 8660);
            SpecChars.Add("&hearts;", 9829);    SpecChars.Add("&hellip;", 8230);    SpecChars.Add("&Iacute;", 205);
            SpecChars.Add("&iacute;", 237);     SpecChars.Add("&Icirc;", 206);      SpecChars.Add("&icirc;", 238);
            SpecChars.Add("&iexcl;", 161);      SpecChars.Add("&Igrave;", 204);     SpecChars.Add("&igrave;", 236);
            SpecChars.Add("&image;", 8465);     SpecChars.Add("&infin;", 8734);     SpecChars.Add("&int;", 8747);
            SpecChars.Add("&Iota;", 921);       SpecChars.Add("&iota;", 953);       SpecChars.Add("&iquest;", 191);
            SpecChars.Add("&isin;", 8712);      SpecChars.Add("&Iuml;", 207);       SpecChars.Add("&iuml;", 239);
            SpecChars.Add("&Kappa;", 922);      SpecChars.Add("&kappa;", 954);      SpecChars.Add("&Lambda;", 923);
            SpecChars.Add("&lambda;", 955);     SpecChars.Add("&lang;", 9001);      SpecChars.Add("&laquo;", 171);
            SpecChars.Add("&larr;", 8592);      SpecChars.Add("&lArr;", 8656);      SpecChars.Add("&lceil;", 8968);
            SpecChars.Add("&ldquo;", 8220);     SpecChars.Add("&le;", 8804);        SpecChars.Add("&lfloor;", 8970);
            SpecChars.Add("&lowast;", 8727);    SpecChars.Add("&loz;", 9674);       SpecChars.Add("&lrm;", 8206);
            SpecChars.Add("&lsaquo;", 8249);    SpecChars.Add("&lsquo;", 8216);     SpecChars.Add("&lt;", 60);
            SpecChars.Add("&macr;", 175);       SpecChars.Add("&mdash;", 8212);     SpecChars.Add("&micro;", 181);
            SpecChars.Add("&middot;", 183);     SpecChars.Add("&minus;", 8722);     SpecChars.Add("&Mu;", 924);
            SpecChars.Add("&mu;", 956);         SpecChars.Add("&nabla;", 8711);     SpecChars.Add("&nbsp;", 32); // Using space instead of Ascii 160
            SpecChars.Add("&ndash;", 8211);     SpecChars.Add("&ne;", 8800);        SpecChars.Add("&ni;", 8715);
            SpecChars.Add("&not;", 172);        SpecChars.Add("&notin;", 8713);     SpecChars.Add("&nsub;", 8836);
            SpecChars.Add("&Ntilde;", 209);     SpecChars.Add("&ntilde;", 241);     SpecChars.Add("&Nu;", 925);
            SpecChars.Add("&nu;", 957);         SpecChars.Add("&Oacute;", 211);     SpecChars.Add("&oacute;", 243);
            SpecChars.Add("&Ocirc;", 212);      SpecChars.Add("&ocirc;", 244);      SpecChars.Add("&OElig;", 338);
            SpecChars.Add("&oelig;", 339);      SpecChars.Add("&Ograve;", 210);     SpecChars.Add("&ograve;", 242);
            SpecChars.Add("&oline;", 8254);     SpecChars.Add("&Omega;", 937);      SpecChars.Add("&omega;", 969);
            SpecChars.Add("&Omicron;", 927);    SpecChars.Add("&omicron;", 959);    SpecChars.Add("&oplus;", 8853);
            SpecChars.Add("&or;", 8744);        SpecChars.Add("&ordf;", 170);       SpecChars.Add("&ordm;", 186);
            SpecChars.Add("&Oslash;", 216);     SpecChars.Add("&oslash;", 248);     SpecChars.Add("&Otilde;", 213);
            SpecChars.Add("&otilde;", 245);     SpecChars.Add("&otimes;", 8855);    SpecChars.Add("&Ouml;", 214);
            SpecChars.Add("&ouml;", 246);       SpecChars.Add("&para;", 182);       SpecChars.Add("&part;", 8706);
            SpecChars.Add("&permil;", 8240);    SpecChars.Add("&perp;", 8869);      SpecChars.Add("&Phi;", 934);
            SpecChars.Add("&phi;", 966);        SpecChars.Add("&Pi;", 928);         SpecChars.Add("&pi;", 960);
            SpecChars.Add("&piv;", 982);        SpecChars.Add("&plusmn;", 177);     SpecChars.Add("&pound;", 163);
            SpecChars.Add("&prime;", 8242);     SpecChars.Add("&Prime;", 8243);     SpecChars.Add("&prod;", 8719);
            SpecChars.Add("&prop;", 8733);      SpecChars.Add("&Psi;", 936);        SpecChars.Add("&psi;", 968);
            SpecChars.Add("&quot;", 34);        SpecChars.Add("&radic;", 8730);     SpecChars.Add("&rang;", 9002);
            SpecChars.Add("&raquo;", 187);      SpecChars.Add("&rarr;", 8594);      SpecChars.Add("&rArr;", 8658);
            SpecChars.Add("&rceil;", 8969);     SpecChars.Add("&rdquo;", 8221);     SpecChars.Add("&real;", 8476);
            SpecChars.Add("&reg;", 174);        SpecChars.Add("&rfloor;", 8971);    SpecChars.Add("&Rho;", 929);
            SpecChars.Add("&rho;", 961);        SpecChars.Add("&rlm;", 8207);       SpecChars.Add("&rsaquo;", 8250);
            SpecChars.Add("&rsquo;", 8217);     SpecChars.Add("&sbquo;", 8218);     SpecChars.Add("&Scaron;", 352);
            SpecChars.Add("&scaron;", 353);     SpecChars.Add("&sdot;", 8901);      SpecChars.Add("&sect;", 167);
            SpecChars.Add("&shy;", 173);        SpecChars.Add("&Sigma;", 931);      SpecChars.Add("&sigma;", 963);
            SpecChars.Add("&sigmaf;", 962);     SpecChars.Add("&sim;", 8764);       SpecChars.Add("&spades;", 9824);
            SpecChars.Add("&sub;", 8834);       SpecChars.Add("&sube;", 8838);      SpecChars.Add("&sum;", 8721);
            SpecChars.Add("&sup;", 8835);       SpecChars.Add("&sup1;", 185);       SpecChars.Add("&sup2;", 178);
            SpecChars.Add("&sup3;", 179);       SpecChars.Add("&supe;", 8839);      SpecChars.Add("&szlig;", 223);
            SpecChars.Add("&Tau;", 932);        SpecChars.Add("&tau;", 964);        SpecChars.Add("&there4;", 8756);
            SpecChars.Add("&Theta;", 920);      SpecChars.Add("&theta;", 952);      SpecChars.Add("&thetasym;", 977);
            SpecChars.Add("&thinsp;", 8201);    SpecChars.Add("&THORN;", 222);      SpecChars.Add("&thorn;", 254);
            SpecChars.Add("&tilde;", 732);      SpecChars.Add("&times;", 215);      SpecChars.Add("&trade;", 8482);
            SpecChars.Add("&Uacute;", 218);     SpecChars.Add("&uacute;", 250);     SpecChars.Add("&uarr;", 8593);
            SpecChars.Add("&uArr;", 8657);      SpecChars.Add("&Ucirc;", 219);      SpecChars.Add("&ucirc;", 251);
            SpecChars.Add("&Ugrave;", 217);     SpecChars.Add("&ugrave;", 249);     SpecChars.Add("&uml;", 168);
            SpecChars.Add("&upsih;", 978);      SpecChars.Add("&Upsilon;", 933);    SpecChars.Add("&upsilon;", 965);
            SpecChars.Add("&Uuml;", 220);       SpecChars.Add("&uuml;", 252);       SpecChars.Add("&weierp;", 8472);
            SpecChars.Add("&Xi;", 926);         SpecChars.Add("&xi;", 958);         SpecChars.Add("&Yacute;", 221);
            SpecChars.Add("&yacute;", 253);     SpecChars.Add("&yen;", 165);        SpecChars.Add("&yuml;", 255);
            SpecChars.Add("&Yuml;", 376);       SpecChars.Add("&Zeta;", 918);       SpecChars.Add("&zeta;", 950);
            SpecChars.Add("&zwj;", 8205);       SpecChars.Add("&zwnj;", 8204);
 
            HtmlToTextReplaceElement[] ReplaceStrings = new HtmlToTextReplaceElement[]
            {
                // Remove HTML Development formatting
                //Replace any white space characters (line breaks, tabs, spaces) with space because browsers inserts space 
                new HtmlToTextReplaceElement(@"\s", " ", HtmlToTextReplaceType.RegEx),
                // Remove repeating speces becuase browsers ignore them 
                new HtmlToTextReplaceElement(@" {2,}", " ", HtmlToTextReplaceType.RegEx),
                /*
                  I'm using .* in my regex from here to match "all" characters. It works here ONLY because I've removed
                  all linebreaks beforehand. If you doesn't done that you MUST replace .* with [\s\đ]* to match all characters
                  in multiple lines
                */
                // Remove HTML comment
                new HtmlToTextReplaceElement(@"<! *--.*?-- *>", " ", HtmlToTextReplaceType.RegEx),
                // Remove the header
                new HtmlToTextReplaceElement(@"< *head( *>| [^>]*>).*< */ *head *>", string.Empty, HtmlToTextReplaceType.RegEx),
                // remove all scripts
                new HtmlToTextReplaceElement(@"< *script( *>| [^>]*>).*?< */ *script *>", string.Empty, HtmlToTextReplaceType.RegEx),
                // remove all styles (prepare first by clearing attributes)
                new HtmlToTextReplaceElement(@"< *style( *>| [^>]*>).*?< */ *style *>", string.Empty, HtmlToTextReplaceType.RegEx),
                // insert tabs in spaces of <td> tags
                new HtmlToTextReplaceElement(@"< *td[^>]*>","\t", HtmlToTextReplaceType.RegEx),
                // insert line breaks in places of <BR> and <LI> tags 
                new HtmlToTextReplaceElement(@"< *(br|li) */{0,1} *>", "\r", HtmlToTextReplaceType.RegEx),
                new HtmlToTextReplaceElement(@"< *(div|tr|p)( *>| [^>]*>)", "\r\r", HtmlToTextReplaceType.RegEx),
                // Remove remaining tags like <a>, links, images, etc - anything thats enclosed inside < > 
                new HtmlToTextReplaceElement(@"<[^>]*>", string.Empty, HtmlToTextReplaceType.RegEx),
                // Replace &nbsp; with whitespace. It is done here because the generated space will be used in
                // whitespace optimizations
                new HtmlToTextReplaceElement(@"&nbsp;", " ", HtmlToTextReplaceType.String),
                // Remove extra line breaks and tabs:
                // Romove any whitespace and tab at and of any line
                new HtmlToTextReplaceElement(@"[ \t]+\r", "\r", HtmlToTextReplaceType.RegEx),
                // Remove whitespace beetween tabs
                new HtmlToTextReplaceElement(@"\t +\t", "\t\t", HtmlToTextReplaceType.RegEx),
                // Remove whitespace begining of a line if followed by a tab
                new HtmlToTextReplaceElement(@"\r +\t", "\r\t", HtmlToTextReplaceType.RegEx),
                // Remove multible tabs following a linebreak with just one tab 
                new HtmlToTextReplaceElement(@"\r\t{2,}", "\r\t", HtmlToTextReplaceType.RegEx),
                // replace over 2 breaks with 2 and over 4 tabs with 4.  
                new HtmlToTextReplaceElement(@"\r{3,}", "\r\r", HtmlToTextReplaceType.RegEx),
                new HtmlToTextReplaceElement(@"\t{4,}", "\t\t\t\t", HtmlToTextReplaceType.RegEx)
            };
            string result = source;
            // run pattern matching
            for (int i = 0; i < ReplaceStrings.Length; i++)
            {
                switch (ReplaceStrings[i].Type)
                {
                    case HtmlToTextReplaceType.String:
                        result = result.Replace(ReplaceStrings[i].Pattern, ReplaceStrings[i].Substitute);
                        break;
                    case HtmlToTextReplaceType.RegEx:
                        result = Regex.Replace(result, ReplaceStrings[i].Pattern, ReplaceStrings[i].Substitute,RegexOptions.IgnoreCase);
                        break;
                }
            }
            // Replace decimal character codes
            AmpCodes = Regex.Matches(result, @"&#\d{1,5};",RegexOptions.IgnoreCase);
            for (int i = AmpCodes.Count - 1; i >= 0; i--)
            {
                AmpStr = AmpCodes[i].Value;
                result = result.Substring(0, AmpCodes[i].Index) +
                    Convert.ToChar(Int32.Parse(AmpStr.Substring(2, AmpStr.Length - 3))) +
                    result.Substring(AmpCodes[i].Index + AmpCodes[i].Length);
            }
            // Replace hexadecimal character codes
            AmpCodes = Regex.Matches(result, @"&#x[0-9a-f]{1,4};",RegexOptions.IgnoreCase);
            for (int i = AmpCodes.Count - 1; i >= 0; i--)
            {
                AmpStr = AmpCodes[i].Value;
                result = result.Substring(0, AmpCodes[i].Index) +
                    Convert.ToChar(Int32.Parse(AmpStr.Substring(3, AmpStr.Length - 4),NumberStyles.AllowHexSpecifier)) +
                    result.Substring(AmpCodes[i].Index + AmpCodes[i].Length);
            }
            // Replace named character codes
            AmpCodes = Regex.Matches(result, @"&\w+;", RegexOptions.IgnoreCase);
            for (int i = AmpCodes.Count - 1; i >= 0; i--)
            {
                if (SpecChars.ContainsKey(AmpCodes[i].Value))
                {
                    result = result.Substring(0, AmpCodes[i].Index) +
                        Convert.ToChar(SpecChars[AmpCodes[i].Value]) +
                        result.Substring(AmpCodes[i].Index + AmpCodes[i].Length);
                }
            }
            // Remove all others
            result = Regex.Replace(result, @"&[^;]*;", string.Empty,RegexOptions.IgnoreCase);
            return result;
        }
    }
    public enum HtmlToTextReplaceType { String, RegEx }
    public class HtmlToTextReplaceElement
    {
        public HtmlToTextReplaceElement() { }
        public HtmlToTextReplaceElement(string Pattern, string Substitute, HtmlToTextReplaceType Type)
        {
            this.Pattern = Pattern;
            this.Substitute = Substitute;
            this.Type = Type;
        }
        public string Pattern;
        public string Substitute;
        public HtmlToTextReplaceType Type;
    }

in addition to the standard ones you have to use this two using in the cs file header:

using System.Text.RegularExpressions;
using System.Globalization;

[Update] Unfortunately two & nbsp ; text in the code converted back to space (due to some problems with the site's engine):
SpecChars.Add(" ", 32);

new HtmlToTextReplaceElement(@" ", " ", HtmlToTextReplaceType.String),

change them back otherwise it will not work correctly

Just what I needed!

paul_t_d

7:42 23 Nov '10

You're a life saver. Changed it a bit to only add one carriage return for each tag, but otherwise exactly what I needed. Thanks for sharing. Hope I can add something as useful sometime...
Sign In·View Thread·Permalink

Improvement to your great piece of work.

PaulTheSDET

5:31 11 Nov '10

When using your code with the System.Web.HttpUtility.HtmlEncode function, I found that the output of the function produced "&amp;lt;=" for "<=". This string of characters became "&lt;=" when using the System.Web.HttpUtility.HtmlDecode function. This of course would then produce "=" when passed through the StripHTML function.

I added this additional Replace action to fix this issue in the area of your code that deals with replacing special characters, just above where you replace the "&bull" value.

                        // replace special characters:
                        result = Regex.Replace(result,
                                    @" ", " ",
                                    RegexOptions.IgnoreCase);

                        // Added for fix
                        result = Regex.Replace(result,
                                    @"&Amp;", "&",
                                    RegexOptions.IgnoreCase);

                        result = Regex.Replace(result,
                                    @"•", " * ",
                                    RegexOptions.IgnoreCase);

This fix will now represent "<" in the result string, to be replaced by the proper value later in the function.

Great function. Big Grin | :-D

My vote of 5

mkdmkd_1

20:05 28 Oct '10

this is usefull for me
Sign In·View Thread·Permalink

My vote of 5

beat_swamp

21:40 2 Aug '10

very useful and work right
Sign In·View Thread·Permalink

Did not work in this case

enjoycrack

22:47 12 Apr '10

the source string has something like <a onmouseover='This is text that has less than and greater than characters <> ... ' >I want to extract text here only</a> How to get the text that is between open/close <a> tags <a>...</a>? 8x Solutions Inc. - Software Outsourcing Company in HCMC, Vietnam
Sign In·View Thread·Permalink

Very nice!!!

JohnAndre

22:49 6 Apr '10

Thank you for all of your efforts. Your solution works well in my project.
Sign In·View Thread·Permalink

Convert HTML to plain Text(Built in function in C#.net 2005)

alaatelfah87

8:42 9 Nov '09

first of all ,Many thanks for this article i found it many useful.
but there is in C#.net 2005 class called Ihtmlclass so when you need to convert html to text you can do
this :
string description="My Html Code";//as <html>

Hi ....

</html>
IHTMLDocument2 doc = new HTMLDocumentClass();

if (doc == null)
{
SystemClass.WriteToLogFile("<br>" + " Faild to Allocate HtmlDocument2 Object");
continue;
}

string b = "<body>";
string endb = "</body>";
if (description.Contains("<body>"))
{
b = endb = "";
}
if (description != System.Web.HttpUtility.HtmlDecode(description))
description = System.Web.HttpUtility.HtmlDecode(description);
doc.clear(); // clear the doc object
doc.write(new object[] { b + description + endb });
doc.close();
description = doc.body.outerText;

i want to ask you if your code do the same what the IHtmlDovument object do ?
And Thanks.

Re: Convert HTML to plain Text(Built in function in C#.net 2005)

jomygeorge

2:35 12 Nov '09

this code above dont work
Sign In·View Thread·Permalink

Re: Convert HTML to plain Text(Built in function in C#.net 2005)

conrad Braam

8:56 10 May '10

It is a pity jomygeourg had enouth time to tel use HttpUtility.HtmlDecode does not do the job, but not enough to tell us all exactly why. So here is why, HtmlEncode converts all the the HTTP characters to their equivalents, and HtmlDecode does the reverse, basically it just changes the protocol, and as this article sets out to do is give you just the text, as is clearly explained, we want to strip out all heading,script, image,links,formatting and so on HttpUtility.HtmlDecode is not doing anything of that nature see http://msdn.microsoft.com/en-us/library/3ykbs7dz.aspx[^]

Oh, and PS, the code in the article seems to only scrape the first panel of a google site. Will have to debug.

Conrad -
The world waits for you to stick your neck out, it's not easy. But once you actually stand up, you do get noticed.
http://www.plcsimulator.org/

thanks

mostafasajjadi

20:03 14 Jul '09

thanks alot dude u make my work so much shorter PEace on U
Sign In·View Thread·Permalink

Thanks

headstones09

6:33 2 Jun '09

Thanks for posting this article.
Sign In·View Thread·Permalink

good article

abutun

11:06 14 May '09

thank you very very much Ahmet BÜTÜN http://www.ahmetbutun.net
Sign In·View Thread·Permalink

thanks, good for plain Text in html mails

yeya8a

17:51 16 Apr '09

Thanks i need something for my html send mails, when the mail is readed in a mail client like horde or squirremail doesn't get the html page until you click the link view on text\html, thats a problem (doesn't happen with hotmail or yahoo), i need to show my content in plain to be readed and then the user takes the decision to view on good presentation in html. Well, Thanks a lot.
Sign In·View Thread·Permalink

please help

pcfacile

8:45 22 Feb '09

i search for visual basic code that i can use in a macro to copy tables to excel sheet... from : http://www.nhl.com/scores/htmlreports/20082009/PL020875.HTM i want to automate the extraction process in my excel macro to do it several times from similar pages with different datas and do a analysist of the data. i want to keep same format as in the web page can you please help me?
Sign In·View Thread·Permalink	1.33/5 (2 votes)

I need help

taminha

4:08 7 Feb '09

how do i change this [A12_B40:50] to [50] using regular expression? that is deleting every thing left to :
Sign In·View Thread·Permalink	1.00/5 (1 vote)

good but a few problems

Member 2458066

14:28 5 Feb '09

First of all, thanks for the code. This does work pretty well, but I have a few issues:

1) If you have script tags at the top and further down inside the body, it matches everything between the start of the first script tag and the end of the last script tag.

You can test this by adding <script></script> to the top of the page and again at the bottom. When you run it you'll end up with an empty string.

The same problem presumably applies to style and head tags (although you shouldn't have a page like that).

2) If you have a textbox in the page with a value in it, it is lost. It would be nice if the value of the textbox could be retained.

3) Similar to #2, it would be nice if you could retain only the selected value in a selection list, rather than including all possible values.

Re: good but a few problems

Martin Ørding-Thomsen

22:14 2 Nov '10

First problem is solved easily by using the non-greedy operator *? in stead of *

result = System.Text.RegularExpressions.Regex.Replace(result,
@"(<script>).*(</script>)",string.Empty,
System.Text.RegularExpressions.RegexOptions.IgnoreCase);

becomes

result = System.Text.RegularExpressions.Regex.Replace(result,
@"(<script>).*?(</script>)",string.Empty,
System.Text.RegularExpressions.RegexOptions.IgnoreCase);

good, but slow

bookerman

4:26 19 Jan '09

this code works to slow for big html files:



for (int index = 0; index < result.Length; index++)

{

	result = result.Replace(breaks, "\r\r");

	result = result.Replace(tabs, "\t\t\t\t");

	breaks = breaks + "\r";

	tabs = tabs + "\t";

}

if comment it or replace with optimized version:



bool found = true;

for (int index = 0; found && index < result.Length; index++)

{

	found = false;

	if (result.Contains(breaks))

		found = true;

	if (result.Contains(tabs))

		found = true;

 

	if (found)

	{

		result = result.Replace(breaks, "\r\r");

		result = result.Replace(tabs, "\t\t\t\t");

		breaks = breaks + "\r";

		tabs = tabs + "\t";

	}

}

function works much faster Shucks | :->

Re: good, but slow

Junlin Xu

7:31 17 Aug '11

should be: while (result.Contains(breaks)) { result = result.Replace(breaks, "\r\r"); } while (result.Contains(tabs)) { result = result.Replace(tabs, "\t\t\t\t"); }
Sign In·View Thread·Permalink