Click here to Skip to main content
15,867,939 members
Articles / Programming Languages / C#
Article

Simple log parsing using MS Log Parser 2.2, in C#.NET

Rate me:
Please Sign up or sign in to vote.
4.87/5 (11 votes)
20 Mar 2006GPL31 min read 141.5K   39   18
A very simple way to parse all sorts of logs like W3C, IIS, WMS etc.

Introduction

At some stage in developing a server side application, I needed a way to check a particular user's download bandwidth. We have lots of client accounts in IIS6 and WMS (Windows Media Server), and each user has got a specific folder (like http://blablabla.com/xxx/, where xxx is a number like 122, 133 etc.). So if someone downloads media content from a specific user's directory, it will get logged in the IIS and WMS logs (IIS will log the download, and WMS logs the streaming bandwidth).

I wrote an NT service in C# to periodically parse the IIS/WMS logs and fetch the bandwidth usage of each user, and log it to a database. The code snippet is simple and easily understandable, and I suggest you download Microsoft Log Parser 2.2 freely from Microsoft, and check the samples and example code and SQL statements. It has got tons of features.

The sample log file I used is as below. I've put in bold, the directory name for which the bandwidth usage will be retrieved. (The text gets wrapped.. just manage guys).

#Version: 1.0
#Fields: date time cs-method cs-uri cs-version c-ip x-distributor-id 
x-ext-status sc-bytes time-taken sc-status cs(User-Agent) cs(Referer) cs(Cookie) 
cs-uri-stem cs-uri-query
2006-03-10 21:19:15.382 GET /Temp/59/Media/movies.wmv HTTP/1.1 
24.90.114.54 2011323 2000101 2018995 7393 206 "NSPlayer/9.0.0.3250 WMFSDK/9.0" 
"-" "-" /Temp/59/Media/movies.wmv -
2006-03-10 21:19:36.068 GET /Temp/59/Media/movies.wmv HTTP/1.1 
24.90.114.54 2011321 10 2020455 27 304 "Windows-Media-Player/9.00.00.3344" "-" 
"-" /Temp/59/Media/movies.wmv -
2006-03-10 21:19:36.084 GET /Temp/59/Media/movies.wmv HTTP/1.0 
206.24.192.232 2011323 121 2020455 1891 200 "Windows-Media-Player/9.00.00.3344" 
"-" "-" /Temp/59/Media/movies.wmv -

Using the code

Download and install Log parser 2.2 from Microsoft (Google it to find the download link.... very strange, it's been hidden somewhere deep in the MS site). After installing it, add a reference to LogParser.dll in the installation directory. Don't forget to put using MSUtil; on top.

C#
//The passing argument "userID" is just the 
//folder name for the bandwidth used need to be retrieved.
//This is just for demonstration. 
//Re structure the SQL query for your needs.

public double ParseW3CLog( string userID )
{

    // prepare LogParser Recordset & Record objects
    ILogRecordset rsLP = null;
    ILogRecord rowLP = null;

    LogQueryClassClass LogParser = null;
    COMW3CInputContextClassClass W3Clog = null;

    double UsedBW = 0;
    int Unitsprocessed;
    double sizeInBytes;

    string strSQL = null;

    LogParser = new LogQueryClassClass();
    W3Clog = new COMW3CInputContextClassClass();

    try
    {
        //W3C Logparsing SQL. Replace this SQL query with whatever 
        //you want to retrieve. The example below 
        //will sum up all the bandwidth
        //Usage of a specific folder with name 
        //"userID". Download Log Parser 2.2 
        //from Microsoft and see sample queries.

        strSQL = @"SELECT SUM(sc-bytes) from C:\\logs" + 
                 @"\\*.log WHERE cs-uri-stem LIKE '%/" + 
                 userID + "/%' ";
    
        // run the query against W3C log
        rsLP = LogParser.Execute(strSQL, W3Clog);
    
        rowLP = rsLP.getRecord();
    
        Unitsprocessed = rsLP.inputUnitsProcessed;

        if (rowLP.getValue(0).ToString() == "0" || 
            rowLP.getValue(0).ToString() == "")
        {
           //Return 0 if an err occured
           UsedBW = 0;
           return UsedBW;
        }

        //Bytes to MB Conversion
        double Bytes = Convert.ToDouble(rowLP.getValue(0).ToString());
        UsedBW = Bytes / (1024 * 1024);

        //Round to 3 decimal places
        UsedBW = Math.Round(UsedBW, 3);
    }
    catch
    {
           throw;
    }

    return UsedBW;
}

Points of Interest

Quite easy, without lines and lines of code... and you will get all sorts of examples from the downloaded toolkit itself. This can be used to retrieve usage stats/bandwidth usage/file type/browser info.. you name it... from almost all types of standard log files (IIS/Mail/WMS/Apache...).

License

This article, along with any associated source code and files, is licensed under The GNU General Public License (GPLv3)


Written By
Engineer ITV Stdios, New York
India India
This member has not yet provided a Biography. Assume it's interesting and varied, and probably something to do with programming.

Comments and Discussions

 
QuestionSystem.Runtime.InteropServices.COMException Pin
satyapraveenav19-Sep-18 7:12
satyapraveenav19-Sep-18 7:12 
AnswerRe: System.Runtime.InteropServices.COMException Pin
satyapraveenav21-Sep-18 8:04
satyapraveenav21-Sep-18 8:04 
QuestionChanging input file Pin
Ramsin23-Apr-14 8:40
Ramsin23-Apr-14 8:40 
QuestionQuestions! Pin
Ramsin19-Apr-14 7:08
Ramsin19-Apr-14 7:08 
AnswerRe: Questions! Pin
Ramsin21-Apr-14 7:15
Ramsin21-Apr-14 7:15 
GeneralRe: Questions! Pin
Aby Watson21-Apr-14 7:26
Aby Watson21-Apr-14 7:26 
'LogQueryClassClass' is not a typo. Thats how its named Smile | :) . Another user has already posted more links for detailed info.... please have a look.

Of course I did compile it and the app was running for a long time. There is no download link associated, as the code is very short, surely someone can copy-paste it to their project.

http://cstruter.com/blog/319[^]

Regards,
Aby
GeneralRe: Questions! Pin
Ramsin22-Apr-14 12:11
Ramsin22-Apr-14 12:11 
GeneralRe: Questions! Pin
Aby Watson22-Apr-14 18:54
Aby Watson22-Apr-14 18:54 
GeneralRe: Questions! Pin
Ramsin23-Apr-14 8:35
Ramsin23-Apr-14 8:35 
GeneralNicely written Pin
CoderPanda30-Jan-14 18:28
professionalCoderPanda30-Jan-14 18:28 
GeneralMore information Pin
Cstruter12-Jun-11 20:50
Cstruter12-Jun-11 20:50 
Generalresult incorrect somethimes Pin
kristofg24-Apr-07 0:16
kristofg24-Apr-07 0:16 
GeneralRe: result incorrect somethimes Pin
speedofspin3-Aug-10 0:15
speedofspin3-Aug-10 0:15 
GeneralI don't get it Pin
Nick D28-Mar-06 22:04
Nick D28-Mar-06 22:04 
GeneralRe: I don't get it Pin
KingJinx1-Apr-06 16:53
KingJinx1-Apr-06 16:53 
GeneralRe: I don't get it Pin
Nick D1-Apr-06 22:56
Nick D1-Apr-06 22:56 
GeneralRe: I don't get it Pin
Bassam Abdul-Baki12-Apr-06 3:53
professionalBassam Abdul-Baki12-Apr-06 3:53 
GeneralRe: I don't get it Pin
pat33321-Oct-08 5:57
pat33321-Oct-08 5:57 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.