Click here to Skip to main content
15,868,141 members
Please Sign up or sign in to vote.
0.00/5 (No votes)
See more:
I need to read a log file which contains one day data. Now I need to parse only the last hours data, whatever the last time may be in the log file... I have written a code which will read from the last line of the file..


<pre lang="java">paths = f1.listFiles();
for(File path:paths) {

ReversedLinesFileReader reverse= new ReversedLinesFileReader(path);

String line;
while ((line = reverse.readLine()) != null) {
    // process the line.
    if(line.contains(Constants.INFO)){
     System.out.println("line contains INFO");                          
    }
    }



Now I need to parse only last one hour data. My last two lines of the log file looks like this.


INFO: May 30 23:59:59 Ignoring load report from EX8103 for DT5243 as it is not assigned to a loading tool.

INFO: May 30 23:59:59 Assignment succeeded: Sat May 31 00:00:01 WST 2014: SUCCESS DT4206


[edit from="OP"]
Sorry for the trouble, I am pasting my code here. This is the code which reads the file backwards.

Java
package com.Test;
import java.io.BufferedReader;
import java.io.File;
import java.io.FileReader;
import java.io.IOException;
import java.text.ParseException;
import java.text.SimpleDateFormat;
import java.util.ArrayList;
import java.util.Date;
import java.util.List;
import java.util.Scanner;
import java.util.regex.Matcher;
import java.util.regex.Pattern;

import org.apache.commons.io.input.ReversedLinesFileReader;








/**
 * Created with IntelliJ IDEA.
 * User: kumarao
 * Date: 24/5/14
 * Time: 3:41 PM
 * To change this template use File | Settings | File Templates.
 */
public class ServerFileProcessor {

	static File f1=null;
    private Date date;
    private static PropertiesReader propertiesReader;
    static File[] paths;
    
    /*public ServerFileProcessor() {
    	propertiesReader = new PropertiesReader();
    	propertiesReader.load("timeGap.properties");
    	
	}
	
*/  
final static Pattern pattern = Pattern.compile(Constants.TIME_PATTERN);

static SimpleDateFormat simpleDateFormat = new SimpleDateFormat("HH:mm:ss");
public static void main() throws IOException {
	propertiesReader = new PropertiesReader();
	propertiesReader.load("Server.properties");
	f1= new File(propertiesReader.getPropertyValue("FOLDER.serverLogs"));
	paths = f1.listFiles();
	for(File path:paths) {
		ReversedLinesFileReader reverse= new ReversedLinesFileReader(path);
    
    String line;
    while ((line = reverse.readLine()) != null) {
        // process the line.
        if(line.contains(Constants.INFO)){
        	Scanner sc = new Scanner(line);
            Date time = null;
            SimpleDateFormat dateFormat = new SimpleDateFormat("HH:mm:ss");
            while(sc.hasNextLine()) {
                final String line1 = sc.nextLine();

                try {
                    Matcher matcher = pattern.matcher(line1);
                    String timeStr = null;
                    while (matcher.find()) {
                        timeStr = matcher.group(1);

                    }
                    time = dateFormat.parse(timeStr);
                    System.out.println(time);
                } catch (ParseException e) {
                    e.printStackTrace();  //To change body of catch statement use File | Settings | File Templates.
                }
            }       	
        }
}
    
    }
	}

}


This is a part of my log file:

SEVERE: number of fids inserted do not match number of fids returned by Transaction Response.  Got:1 expected: 0
INFO: Jun 13 23:59:51 Not updating DT6023 with delay information: 02_Crib for DT6023 from 23:59:50 13 Jun 14 to 00:43:50 14 Jun 14 
Jun 13, 2014 11:59:52 PM org.geotools.data.wfs.v1_0_0.WFSTransactionState commit
SEVERE: number of fids inserted do not match number of fids returned by Transaction Response.  Got:1 expected: 0
INFO: Jun 13 23:59:52 DT5204 ASSIGNED to DUMP SABM at TSFROM_Skyway_Ext into processor TSFROM_Skyway_Ex. 
INFO: Jun 13 23:59:52 Assignment succeeded: Fri Jun 13 23:59:53 WST 2014: SUCCESS DT5204 
INFO: Jun 13 23:59:52 DT4155 ASSIGNED to DUMP SADE at VOK_V0002 into processor VOK_V0002. 
INFO: Jun 13 23:59:52 Assignment succeeded: Fri Jun 13 23:59:52 WST 2014: SUCCESS DT4155 
INFO: Jun 13 23:59:53 Low Z 588,840.96 m, 7,549,399 m, 546.82 m corrected to 551.2 m for EX8105 
INFO: Jun 13 23:59:54 Cycle fuel usage estimate was inaccurate: DT5210 estimated fuel amount 0 l was compared to 92.07 l 
INFO: Jun 13 23:59:55 Low Z 590,262.02 m, 7,553,127.96 m, 545.39 m corrected to 546.37 m for WL7024 
Jun 13, 2014 11:59:55 PM org.geotools.data.wfs.v1_0_0.WFSTransactionState commit
SEVERE: number of fids inserted do not match number of fids returned by Transaction Response.  Got:1 expected: 0
INFO: Jun 13 23:59:55 TruckLoadMessage DT5205 VOK_01_B EX8101 UNKNOWN 
Jun 13, 2014 11:59:57 PM org.geotools.data.wfs.v1_0_0.WFSTransactionState commit
SEVERE: number of fids inserted do not match number of fids returned by Transaction Response.  Got:1 expected: 0
Jun 13, 2014 11:59:57 PM org.geotools.data.wfs.v1_0_0.WFSTransactionState commit
SEVERE: number of fids inserted do not match number of fids returned by Transaction Response.  Got:1 expected: 0
Jun 13, 2014 11:59:58 PM org.geotools.data.wfs.v1_0_0.WFSTransactionState commit
SEVERE: number of fids inserted do not match number of fids returned by Transaction Response.  Got:1 expected: 0
Jun 13, 2014 11:59:59 PM org.geotools.data.wfs.v1_0_0.WFSTransactionState commit
SEVERE: number of fids inserted do not match number of fids returned by Transaction Response.  Got:1 expected: 0
PERFORMANCE: Jun 13 23:59:59 Service 'sendDelayRequest' RPC took too long: 150,002 millis [ServiceProviderProxy] Thread pool-1-thread-1140
INFO: Jun 13 23:59:59 Updating WL7106 with active delay information: {startTime=Fri Jun 13 23:24:57 WST 2014, timestamp=Fri Jun 13 23:24:59 WST 2014, duration=-1, interfaceName=Assignment, delayId=178, machine=WL7106} 
Jun 13, 2014 11:59:59 PM org.geotools.data.wfs.v1_0_0.WFSTransactionState commit
SEVERE: number of fids inserted do not match number of fids returned by Transaction Response.  Got:1 expected: 0


I need to read only the last hour log file. Whatever the last hour may be. I am getting this value in my console now..

Thu Jan 01 00:00:04 IST 1970
Thu Jan 01 00:00:04 IST 1970
Thu Jan 01 00:00:04 IST 1970
Thu Jan 01 00:00:04 IST 1970
Thu Jan 01 00:00:04 IST 1970
Thu Jan 01 00:00:03 IST 1970
Thu Jan 01 00:00:03 IST 1970
Thu Jan 01 00:00:00 IST 1970
Thu Jan 01 00:00:00 IST 1970
Thu Jan 01 00:00:00 IST 1970
Thu Jan 01 00:00:00 IST 1970
Thu Jan 01 00:00:01 IST 1970
Thu Jan 01 00:00:00 IST 1970
Thu Jan 01 00:00:00 IST 1970


And that too many .... Please help.

[/edit]
Posted
Updated 14-Jul-14 0:47am
v2
Comments
Richard MacCutchan 14-Jul-14 5:21am    
So you know that the last hour was 23, just search the file backwards until you reach one containing 22 and then start reading forwards from there.
Member 10885013 14-Jul-14 5:42am    
I need to create a service which automatically parses just the last hour data for any log file. Above is just an example sir.
Richard MacCutchan 14-Jul-14 5:55am    
OK, but you need to be much more specific about what you need help with. What do you mean by a "service", what is the format of other log files that you may action, what do you want to do with the data you extract, etc.?
Member 10885013 14-Jul-14 6:01am    
Content moved to question.
Richard MacCutchan 14-Jul-14 6:51am    
Obviously something is not working with your matching and parsing. You need to use your debugger to step through the code to see exactly what values your code is receiving from these methods and why they are not correct.

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)



CodeProject, 20 Bay Street, 11th Floor Toronto, Ontario, Canada M5J 2N8 +1 (416) 849-8900