Click here to Skip to main content
15,896,549 members
Please Sign up or sign in to vote.
0.00/5 (No votes)
See more:
How to read a gz file(contains billion lines)using java from bottom to top while reading itself.

Thanks in advance.
Posted
Comments
Richard MacCutchan 2-Jan-15 8:12am    
Your question is not clear, please add more detail. You should also understand that .gz files are compressed and thus do not contain "lines", just bytes.
Angel Roy 5-Jan-15 0:11am    
sorry text file

1 solution

Decompress the ... well lines I guess ...

http://examples.javacodegeeks.com/core-java/io/fileinputstream/decompress-a-gzip-file-in-java-example/[^]

Then you might be able to use the GZIPInputStream object right away and work with what comes in...

Have fun!
 
Share this answer
 
Comments
Angel Roy 16-Jan-15 5:46am    
ya I decompress the file and get the lines from bottom while reading a single line. but it's take too much of time. Is there any simple way? Thanks in advance
My code is,
//Tail
/*
* To change this template, choose Tools | Templates
* and open the template in the editor.
*/
package packetsend;

import java.io.ByteArrayOutputStream;
import java.io.File;
import java.io.IOException;
import java.io.RandomAccessFile;
import java.io.UnsupportedEncodingException;
import java.nio.ByteBuffer;
import java.nio.channels.FileChannel;

/**
*
* @author rega
*/
public class Tail {

private static final int BUFFER_SIZE = 8192;
private final FileChannel channel;
private long filePos;
private ByteBuffer buf;
private int bufPos;
private byte lastLineBreak = ' ';
private ByteArrayOutputStream baos = new ByteArrayOutputStream();

public Tail(String path) throws IOException {
RandomAccessFile raf = new RandomAccessFile(path, "rw");
channel = raf.getChannel();
filePos = raf.length();
}

//Pointing to last line and do reverse of it
public String readLine() throws IOException {
while (true) {
if (bufPos < 0) {
if (filePos == 0) {
if (baos == null) {
return null;
}
String line = bufToString();
baos = null;
return line;
}

long start = Math.max(filePos - BUFFER_SIZE, 0);
long end = filePos;
long len = end - start;

buf = channel.map(FileChannel.MapMode.READ_ONLY, start, len);
bufPos = (int) len;
filePos = start;
}

while (bufPos-- > 0) {
byte c = buf.get(bufPos);
if (c == '\r' || c == '\n') {
if (c != lastLineBreak) {
lastLineBreak = c;
continue;
}
lastLineBreak = c;
return bufToString();
}
baos.write(c);
}
}
}
//Reversing ByteArray o/p and converts to String

private String bufToString() throws UnsupportedEncodingException {
if (baos.size() == 0) {
return "";
}

byte[] bytes = baos.toByteArray();
for (int i = 0; i < bytes.length / 2; i++) {
byte t = bytes[i];
bytes[i] = bytes[bytes.length - i - 1];
bytes[bytes.length - i - 1] = t;
}

baos.reset();
return new String(bytes);
}
}

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)



CodeProject, 20 Bay Street, 11th Floor Toronto, Ontario, Canada M5J 2N8 +1 (416) 849-8900