Click here to Skip to main content
15,867,686 members
Please Sign up or sign in to vote.
3.00/5 (1 vote)
See more:
Hello,

I have a doubt about my project "log analysis".Log Analysis means, developing a new tool for analyze the logs.The log files have different formats.So, the first step browse alog file from the directory and extract the fields of log file.How to extract the fields from a log file? Any help plz...
Posted
Comments
Jochen Arndt 30-Jan-14 9:36am    
I would use regular expressions.
Richard MacCutchan 30-Jan-14 12:21pm    
You could, as Jochen suggests, use regexes. But if the data is in different formats you may find it better to write your own parser, which can tokenise the different formats.
Sergey Alexandrovich Kryukov 30-Jan-14 13:29pm    
Too fuzzy question. If we knew how typical logs look, you might get some more useful advice...
—SA
Member 10560011 30-Jan-14 21:17pm    
Sample log file :
216.148.244.91 - - [11/Mar/2005:11:05:27 -0500] "GET /tracker.html?retailer=Test+Retailer&order=xyz123&uniqueId=12345&sku=sku-00005&prodName=the%20greatest%20product%20in%20all%20the%20world&qty=1&unitPrice=15.25 HTTP/1.0" 200 - "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"

1 solution

Depends on what kind of log file. What kind of fields etc.

If it is like an IIS log or event log then use something like Log Parser[^]. Excellent tool.

There are many tools like Log Parser Lizard[^] that is already there to read various types of log files if you do not want to use command line tools like LogParser.

Hope it helps.
 
Share this answer
 
Comments
Member 10560011 30-Jan-14 20:46pm    
Thanks.. these are already existing tool.But am going to develop a new tool for my project.So for example analysis a error log file(the fields are date,time,ip address,message)how these fields are extracting in c++ qt programming.The 1st step is extracting the fields of log file and displays then based on the condition generate the report.
Member 10560011 30-Jan-14 21:18pm    
Sample log file:
216.148.244.91 - - [11/Mar/2005:11:05:27 -0500] "GET /tracker.html?retailer=Test+Retailer&order=xyz123&uniqueId=12345&sku=sku-00005&prodName=the%20greatest%20product%20in%20all%20the%20world&qty=1&unitPrice=15.25 HTTP/1.0" 200 - "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
CoderPanda 30-Jan-14 23:37pm    
Thanks for the details. I still maintain my answer. I think for inspiration you can download the log parser lizard tool and see how the functionality works as your requirements are also similar.

Here is how I would do it:
1. List down all possible fields/columns in your log file.
2. Build a C++/C#/whatever app. Inside the app build log parser queries based on the columns and conditions selected by the user on the UI.
3. Execute those queries against the desired files and generate output in form of text or CSV files.
4. Now use the output csv to display whatever report you like to.

Hope it helps.
Member 10560011 31-Jan-14 0:09am    
Thanks for the help.If there any code for this application?
CoderPanda 31-Jan-14 0:19am    
I do not have full fledged working app for you. However this link should serve as a good starting point. Also please do read a bit about log parser, it is very easy and almost like an industry standard when it comes to log parsing.
http://www.codeproject.com/Articles/13504/Simple-log-parsing-using-MS-Log-Parser-2-2-in-C-NE

If the answers are useful to you or at least provide a starting point in your solution then please up-vote them and encourage participation.

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)



CodeProject, 20 Bay Street, 11th Floor Toronto, Ontario, Canada M5J 2N8 +1 (416) 849-8900