Click here to Skip to main content
15,886,518 members
Please Sign up or sign in to vote.
0.00/5 (No votes)
See more:
I am writing a program in java to merge any number of files (Only files not directories)into a single file and split them when ever needed.

here is the code I have written.


package com.vim.jy;
import java.io.BufferedOutputStream;
import java.io.DataInputStream;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileOutputStream;

public class MergeFile
{
	FileInputStream fis;
	FileOutputStream fos;
	DataInputStream dis;
	BufferedOutputStream bos;
	File file=new File("D:\\Test\\Input\\");
	File[] file_Names;
	int i=0, ltc, file_Size, count_Files; // ltc=Length to copy till how many bytes 
	byte[] rw_Buffer,con_Str_Bt_Arr;
	int size;
	MergeFile() throws Exception
	{	
		System.out.println("Enter into MergeFile  method");
		file_Names=file.listFiles();
		if(file_Names!=null)
		{
			System.out.println("got file names frm "+ file+" folder ");
		}
		
		count_Files=file_Names.length;
		System.out.println("the total number of files in the path "+ file +" : "+count_Files);
		System.out.println("File names: ");
		
		for (int i = 0; i < file_Names.length; i++)
		{
			System.out.println(file_Names[i]+"\t");
		}
		
		if(count_Files!=0)
		{
			System.out.println("got file number  \n");
		}
		
		
		System.out.println("The number of files in the input folder to merge  "+count_Files + "\n");
		
		fos= new FileOutputStream("D:\\Test\\Output\\Out.dat"); // outputs file file location stores
		if(fos!=null)
		{
			System.out.println("fos created "+fos);
		}
		
		
		bos=new BufferedOutputStream(fos);
		if(bos!=null)
		{
			System.out.println("bos created "+bos);
		}
		
		
		size=0;
		rw_Buffer=new byte[999000]; // read and write buffer size
		if(rw_Buffer!=null)
		{
				System.out.println("byte array created "+rw_Buffer);
		}
		
		
		
		int sl;
		while(count_Files>0)
		{
			System.out.println("Entring into while loop");
			String scrFile=file_Names[i].getName();
			System.out.println("<<<<<----->>"+scrFile);

		
			if(scrFile!=null)
			{
				System.out.println("the file name is "+scrFile);
			}
		
			fis=new FileInputStream("D:\\Test\\Input\\"+scrFile);
			if(fis!=null)
			{
				System.out.println("fis is "+fis);
			}
			
			
			dis=new DataInputStream(fis);
			if(dis!=null)
			{
				System.out.println("dis is "+dis);
			}
			
			
			sl=scrFile.length();
			if(sl!=0)
			{
				System.out.println("sl is "+sl);
			}
			
			
			bos.write(sl);
			if(bos!=null)
			{
				System.out.println("bos is written   "+bos);
			}
			else
			{
				System.out.println("bos not  written"+bos);
			}
			con_Str_Bt_Arr=scrFile.getBytes();   //convetting file name in string to byte array 
			if(con_Str_Bt_Arr!=null)
			{
				System.out.println("con_Str_Bt_Arr soruce file is conveted to bytes   "+con_Str_Bt_Arr);
			}
			else
			{
				System.out.println("con_Str_Bt_Arr soruce file not conveted to bytes   "+con_Str_Bt_Arr);
			}
			ltc=con_Str_Bt_Arr.length;			// name lenght
			
			bos.write(con_Str_Bt_Arr,0,ltc);		//adding file name to *.dat file
			file_Size=dis.available();		//content size in file 
			bos.write(file_Size);				//adding content size
			size=size+file_Size;
			System.out.println("----------->>>>"+size);
			file_Size=dis.available();
			dis.readFully(rw_Buffer,0,file_Size);		//reading data from files in bytes
			System.out.println("->>> dis.readFully");
			bos.write(rw_Buffer,0,file_Size);			//writing data to files in bytes
			size=size+file_Size;
			count_Files--;
			i++;
		}
		close();  
		
		
		return;
	}
	public void close() throws Exception
	{
		bos.flush();
		bos.close();
		fos.close();
		
	}
	public static void main(String[] args) throws Exception
	{
		System.out.println("enter main \n");
		new MergeFile();
		System.out.println("exit main  \n");
	}
}


the issue is that when ever I execute this program for files of large size, it is throwing an exception

XML
Exception in thread "main" java.lang.IndexOutOfBoundsException
    at java.io.FileInputStream.readBytes(Native Method)
    at java.io.FileInputStream.read(Unknown Source)
    at java.io.DataInputStream.readFully(Unknown Source)
    at com.vim.jy.MergeFile.<init>(MergeFile.java:128)
    at com.vim.jy.MergeFile.main(MergeFile.java:150)


or memory heap...... (something)

how to fix this??

after this, I need to split the files to the original files.
All this should be done in byte streams

I just don't want to ignore the exception. I need to fix this excetption, memory heap in eclipse (community version) and the reason for this.

* this memory heap ...(something) exception comes when operating on large files of total size (scrFile size) >800KB and 1mb<<
Posted
Updated 28-Oct-10 20:12pm
v2
Comments
E.F. Nijboer 29-Oct-10 12:49pm    
What I do notice this line twice: size=size+file_Size;
You could look into this using a debugger and have a look at the variables during execution. Maybe you can add when it goes wrong or maybe will already see why the exception is thrown.

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)



CodeProject, 20 Bay Street, 11th Floor Toronto, Ontario, Canada M5J 2N8 +1 (416) 849-8900