Click here to Skip to main content
Click here to Skip to main content

Simple log parsing using MS Log Parser 2.2, in C#.NET

, 20 Mar 2006 GPL3
Rate this:
Please Sign up or sign in to vote.
A very simple way to parse all sorts of logs like W3C, IIS, WMS etc.

Introduction

At some stage in developing a server side application, I needed a way to check a particular user's download bandwidth. We have lots of client accounts in IIS6 and WMS (Windows Media Server), and each user has got a specific folder (like http://blablabla.com/xxx/, where xxx is a number like 122, 133 etc.). So if someone downloads media content from a specific user's directory, it will get logged in the IIS and WMS logs (IIS will log the download, and WMS logs the streaming bandwidth).

I wrote an NT service in C# to periodically parse the IIS/WMS logs and fetch the bandwidth usage of each user, and log it to a database. The code snippet is simple and easily understandable, and I suggest you download Microsoft Log Parser 2.2 freely from Microsoft, and check the samples and example code and SQL statements. It has got tons of features.

The sample log file I used is as below. I've put in bold, the directory name for which the bandwidth usage will be retrieved. (The text gets wrapped.. just manage guys).

#Version: 1.0
#Fields: date time cs-method cs-uri cs-version c-ip x-distributor-id 
x-ext-status sc-bytes time-taken sc-status cs(User-Agent) cs(Referer) cs(Cookie) 
cs-uri-stem cs-uri-query
2006-03-10 21:19:15.382 GET /Temp/59/Media/movies.wmv HTTP/1.1 
24.90.114.54 2011323 2000101 2018995 7393 206 "NSPlayer/9.0.0.3250 WMFSDK/9.0" 
"-" "-" /Temp/59/Media/movies.wmv -
2006-03-10 21:19:36.068 GET /Temp/59/Media/movies.wmv HTTP/1.1 
24.90.114.54 2011321 10 2020455 27 304 "Windows-Media-Player/9.00.00.3344" "-" 
"-" /Temp/59/Media/movies.wmv -
2006-03-10 21:19:36.084 GET /Temp/59/Media/movies.wmv HTTP/1.0 
206.24.192.232 2011323 121 2020455 1891 200 "Windows-Media-Player/9.00.00.3344" 
"-" "-" /Temp/59/Media/movies.wmv -

Using the code

Download and install Log parser 2.2 from Microsoft (Google it to find the download link.... very strange, it's been hidden somewhere deep in the MS site). After installing it, add a reference to LogParser.dll in the installation directory. Don't forget to put using MSUtil; on top.

//The passing argument "userID" is just the 
//folder name for the bandwidth used need to be retrieved.
//This is just for demonstration. 
//Re structure the SQL query for your needs.

public double ParseW3CLog( string userID )
{

    // prepare LogParser Recordset & Record objects
    ILogRecordset rsLP = null;
    ILogRecord rowLP = null;

    LogQueryClassClass LogParser = null;
    COMW3CInputContextClassClass W3Clog = null;

    double UsedBW = 0;
    int Unitsprocessed;
    double sizeInBytes;

    string strSQL = null;

    LogParser = new LogQueryClassClass();
    W3Clog = new COMW3CInputContextClassClass();

    try
    {
        //W3C Logparsing SQL. Replace this SQL query with whatever 
        //you want to retrieve. The example below 
        //will sum up all the bandwidth
        //Usage of a specific folder with name 
        //"userID". Download Log Parser 2.2 
        //from Microsoft and see sample queries.

        strSQL = @"SELECT SUM(sc-bytes) from C:\\logs" + 
                 @"\\*.log WHERE cs-uri-stem LIKE '%/" + 
                 userID + "/%' ";
    
        // run the query against W3C log
        rsLP = LogParser.Execute(strSQL, W3Clog);
    
        rowLP = rsLP.getRecord();
    
        Unitsprocessed = rsLP.inputUnitsProcessed;

        if (rowLP.getValue(0).ToString() == "0" || 
            rowLP.getValue(0).ToString() == "")
        {
           //Return 0 if an err occured
           UsedBW = 0;
           return UsedBW;
        }

        //Bytes to MB Conversion
        double Bytes = Convert.ToDouble(rowLP.getValue(0).ToString());
        UsedBW = Bytes / (1024 * 1024);

        //Round to 3 decimal places
        UsedBW = Math.Round(UsedBW, 3);
    }
    catch
    {
           throw;
    }

    return UsedBW;
}

Points of Interest

Quite easy, without lines and lines of code... and you will get all sorts of examples from the downloaded toolkit itself. This can be used to retrieve usage stats/bandwidth usage/file type/browser info.. you name it... from almost all types of standard log files (IIS/Mail/WMS/Apache...).

License

This article, along with any associated source code and files, is licensed under The GNU General Public License (GPLv3)

Share

About the Author

Aby Watson
Engineer ITV Stdios, New York
India India
No Biography provided

Comments and Discussions

 
QuestionChanging input file PinmemberRamsin23-Apr-14 9:40 
QuestionQuestions! PinmemberRamsin19-Apr-14 8:08 
AnswerRe: Questions! PinmemberRamsin21-Apr-14 8:15 
GeneralRe: Questions! PinmemberAby Watson21-Apr-14 8:26 
GeneralRe: Questions! PinmemberRamsin22-Apr-14 13:11 
GeneralRe: Questions! [modified] PinmemberAby Watson22-Apr-14 19:54 
GeneralRe: Questions! PinmemberRamsin23-Apr-14 9:35 
GeneralNicely written PinprofessionalBraj Panda30-Jan-14 19:28 
GeneralMore information PinmemberCstruter12-Jun-11 21:50 
Generalresult incorrect somethimes Pinmemberkristofg24-Apr-07 1:16 
GeneralRe: result incorrect somethimes Pinmemberspeedofspin3-Aug-10 1:15 
GeneralI don't get it PinmemberNick D28-Mar-06 23:04 
GeneralRe: I don't get it PinmemberKingJinx1-Apr-06 17:53 
GeneralRe: I don't get it PinmemberNick D1-Apr-06 23:56 
GeneralRe: I don't get it PinmemberBassam Abdul-Baki12-Apr-06 4:53 
GeneralRe: I don't get it Pinmemberpat33321-Oct-08 6:57 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

| Advertise | Privacy | Terms of Use | Mobile
Web01 | 2.8.1411023.1 | Last Updated 20 Mar 2006
Article Copyright 2006 by Aby Watson
Everything else Copyright © CodeProject, 1999-2014
Layout: fixed | fluid