HTML Tag Stripper

• Download demo project - 26.0 KB
• Download source - 4.2 KB

Introduction

This article explains a simple method of stripping HTML tags that is similar to the PHP strip_tags() function. This is usually useful in CMS systems where you need to store the text-only version of, for example, an article in order to allow a full-text search through all articles.

Background

Stripping the tags, in this problem's context, means keeping only the visible text of an HTML document or HTML fragment. This means excluding all HTML comments and all HTML <script>, <style> and <noscript> blocks.

I must also mention the fact that the text resulting from this stripping can be processed even more by replacing named HTML entities such as ", &, ©,  , etc. and unnamed HTML entities such as ţ with their corresponding characters. Just set the method's respective parameters, i.e. replaceNamedEntities and replaceNumberedEntities, to true. Bear in mind, however, that these can slow the execution time down significantly.

Using the Code

There is only one method involved in this operation. I called it, without any inspiration, HtmlStripTags. It accepts three parameters:

• htmlContent: the HTML content to process
• replaceNamedEntities: whether to replace the HTML named entities such as   and others
• replaceNumberedEntities: whether to replace the HTML numbered entities, i.e. Unicode HTML representations such as ţ

public static string HtmlStripTags(string htmlContent, 
    bool replaceNamedEntities, bool replaceNumberedEntities)
{
    if (htmlContent == null)
        return null;
    htmlContent = htmlContent.Trim();
    if (htmlContent == string.Empty)
        return string.Empty;

    int bodyStartTagIdx = htmlContent.IndexOf("<body", 
        StringComparison.CurrentCultureIgnoreCase);
    int bodyEndTagIdx = htmlContent.IndexOf("</body>", 
        StringComparison.CurrentCultureIgnoreCase);

    int startIdx = 0, endIdx = htmlContent.Length - 1;
    if (bodyStartTagIdx >= 0)
        startIdx = bodyStartTagIdx;
    if (bodyEndTagIdx >= 0)
        endIdx = bodyEndTagIdx;

    bool insideTag = false,
        insideAttributeValue = false,
        insideHtmlComment = false,
        insideScriptBlock = false,
        insideNoScriptBlock = false,
        insideStyleBlock = false;
    char attributeValueDelimiter = '"';

    StringBuilder sb = new StringBuilder(htmlContent.Length);
    for (int i = startIdx; i <= endIdx; i++)
    {

        // html comment block
        if (!insideHtmlComment)
        {
            if (i + 3 < htmlContent.Length &&
                htmlContent[i] == '<' &&
                ...
                ...
                ...

Points of Interest

I avoided using Regular Expressions in order to achieve maximum performance. RegExs are not yet .NET Framework's strongest point. Moreover, I considered this task simple enough to not require such a universal tool. I ran benchmark tests comparing this implementation to another one presented here at CodeProject.com, Covert HTML to Plain Text, which uses mainly Regular Expressions. I found that when parsing large HTML contents (80+ KB) without replacing any HTML entities, it could yet give 5x times better performance, i.e. ~ 5 microseconds on a Intel Core Duo @ 1,83GHz with 1GB RAM. Of course, it is less elegant than using Regular Expressions. I simply maximized the performance.

History

• 18 July, 2007 -- Original version posted
• 8 November, 2007 -- Article content and downloads updated

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)