Click here to Skip to main content
15,120,739 members
Articles / Logging
Technical Blog
Posted 18 Sep 2012

Tagged as


5 bookmarked

How To Create A Logfile Viewer in 15 Minutes

Rate me:
Please Sign up or sign in to vote.
5.00/5 (2 votes)
18 Sep 2012CPOL4 min read
This post will show you how to create a Logfile Viewer in 15 minutes

Last week, I had to use a customers tool which starts RTSP sessions in a video on demand network. This tool dumped its logging information into a text file which I had to analyze after each run. The logfile got bigger and bigger and it became more and more inconvenient to find the interesting places. And if that wasn't enough, the log contained a lot of message lines that were not useful (at least for my job).

To get an idea about the amount of data : the tool produced ~ 500 lines per second.

So I decided to create a little FlowSheet which should support me a little bit.


To give you an idea how the original data looks like, here's a little excerpt:


The goals were clear:

  1. It should not display all messages lines, only the newly added ones
  2. I'd like to control the update in the viewer
  3. Specific messages should be filtered out completely
  4. I didn't want to see all parts of the single message, only time, severity, text and the stb part.
  5. Specific messages should appear color-coded.

Here's the result:

Image 3

As you can see, I have split the whole thing into several macros.


Image 4

Remember my first design goal? I want to see only new message lines.
My 'problem' is that the operator 'filesource0' loads the complete file, so I need to cut the resulting table into 2 pieces. Section 3 shows how to do that. The 'segrows0' operator gets the complete table and a value into the 'numLines' input. This value comes from a state variable which will be updated after each load with the total number of message lines. Section 4 removes all leading and trailing white space characters (spaces, tabs, CR and LF characters) from all lines. And last but not least, all empty lines will be removed in the macro 'RemoveEmptyLines'.


Image 5

This is the resulting table from macro 'ProcessLog' in the 'Auto Value Watch'.
The raw text was transformed in a simple table (vector) without a header.
The table now contains 5001 rows.


Image 6

Implementation of my third design goal. It throws out all lines which contains a specific keyword.
The actual filter work is done in the loop.

The loop will be repeated 6 times, one time for every row in the feeding table. The table with the lines is routed in a state input - it gets reused and will be reduced (hopefully) in every iteration of the loop.


Image 7

This is the resulting table from macro 'Filter' in the 'Auto Value Watch'.
The table now contains 2461 rows.


Image 8

We're getting close. We have reduced the number of lines, but the format of the lines is still the same. As I stated before, I want to see only columns containing time, severity, text and the stb info.

Let's check what happens here. The operator 'strsplit0' chops the incoming lines into 2 pieces. It uses '[]' as the delimiter.

2012-01-19 10:05:45.681 DEF INFO 00080003 [] 40,STBS-1d90369840737: Key code 3/RCC_DOWN down 

The outbound connector 'data' delivers...

2012-01-19 10:05:45.681 DEF INFO 00080003 the next macro 'ProcessData'.


ProcessData takes advantage of the powerful 'csvimport' to convert the string into the first piece of our desired destination table.

Outbound : ProcessData


ProcessText splits the incoming text again into two parts. The operator 'joincols0' creates the destination table for us. It takes the table from 'ProcessData' and adds two new columns to it. The names of the inbound connectors will be used to set the columns titles. Here's the result:

Image 10

Hmm, what's missing? You're right. The color coding of specific rows will be applied in the macro 'ColorCoder'.


Image 11

This is the resulting table from macro 'PrepareForOutput' in the 'Auto Value Watch'.
The table is fully formatted, it has headers and the color coding was also applied.


Macro GUI

One thing is still missing. I'd like to have a button on my dashboard to initiate the update process. This is an easy task. I use 'buildinput0' which creates the desired button for me. The final step is used to glue the properly formatted table together with my button and pass the result to the dashboard.


Looking behind, I would say creating the FlowSheet hasn't saved me any time.
It took me 10 times longer to write this article. :)


This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)


About the Author

Alexander Golde
Software Developer (Senior)
Germany Germany
No Biography provided

Comments and Discussions

QuestionTwo meta-questions... Pin
Jens Theisen19-Sep-12 11:53
MemberJens Theisen19-Sep-12 11:53 
AnswerRe: Two meta-questions... Pin
Alexander Golde19-Sep-12 21:37
MemberAlexander Golde19-Sep-12 21:37 
GeneralRe: Two meta-questions... Pin
Jens Theisen20-Sep-12 1:29
MemberJens Theisen20-Sep-12 1:29 
GeneralMy vote of 5 Pin
Kanasz Robert19-Sep-12 5:47
professionalKanasz Robert19-Sep-12 5:47 
GeneralRe: My vote of 5 Pin
Alexander Golde19-Sep-12 8:11
MemberAlexander Golde19-Sep-12 8:11 
Thanks. The company just released a free community edition to work with. It's worth a look.
AnswerLooks good... Pin
Sandeep Mewara17-Sep-12 21:38
mvaSandeep Mewara17-Sep-12 21:38 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.