Click here to Skip to main content
15,849,130 members
Please Sign up or sign in to vote.
5.00/5 (1 vote)
See more:
Hi,

1. Which is better way to process the files (File Stream / Memory Stream)?
2. I think byte array decrease the process speed. is this true?

I need your valuable inputs to make the following code more efficient

Thanks in Advance
C#
//split the array into multiple array and stored it in the list for analysis
 private void Split()
            {
                byte[] content = ReadFile(test.txt);
                List<string[]> record = new List<string[]>();
                int splitLength = 130;
                byte[] spiltResult = new string[130];
                for(int i=0; i<content.Length;i = i+spitLength)
                {
                    Array.Copy(content,i,spiltResult,0,splitLength);
                    record.Add(spiltResult);
                }
             content =nuul;
            }
//Returns the byte array of the given file
//Is this a right way to do? i need more efficient code
//Which is better File Stream/Memory Stream
 private byte[] ReadFile(string filePath)
            {
            FileStream fs = new FileStream(filePath, FileMode.Open, ileAccess.Read);
            int length = Convert.ToInt32(fs.Length);
            byte[] data = new byte[length];
            fs.Read(data, 0, length);
            fs.Close();   
            return data;
            }


Regards,
Sowraaj
Posted
Updated 21-Nov-12 3:51am
v3
Comments
Herman<T>.Instance 21-Nov-12 9:00am    
how to measure decrease in process speed?
codeninja-C# 21-Nov-12 9:07am    
Hi,

The byte array size is more than 10,00,000. i thought it probably occupies more memory space so that i come to the decision of worst process. What is the certain way to calculate the program process

Thanks for your responce
S@53K^S 21-Nov-12 10:02am    
I would suggest to use file stream over Memory stream.typically both inherit from the Stream class,but I guess file stream is better equipped to handle files than memory stream.I did not understand your second question clearly
codeninja-C# 21-Nov-12 23:38pm    
Hi,

My second question is based on the array size. I converted the file stream into byte array. I thought when the array size is increasing, it will decrease the program process. Is it true?

and i also have another question which is based on array split
How can i store the splitted array into list from Memory Stream/ File Stream? (With out converting it into the byte array)
S@53K^S 26-Nov-12 13:09pm    
Array is typically value based and is stored in the stack so any operations that you perform on the array increases the memory usage, and by decreasing program process you mean responsiveness of the program?

Sowraaj,

The size of your file is just as important as the way that you read it in. When you're dealing with smaller files, you can read the whole file in and not worry too much about the method. However, when you're dealing with much larger files, say 10Mb and up, its a concern about what you're doing. It may be best to read them in incrementally. Say 100K at a time, process that bit of the file and then read in the next 100K disgarding the previous file contents from memory.

ASM
byte[] data = new byte[1024 * 100];
while(fs.Read(data, 0, length) > 0)
{ 
     Process File Contents Here
}




Hogan
 
Share this answer
 
Comments
codeninja-C# 21-Nov-12 9:25am    
Hi,
I am processing only a 1MB file
Thanks
If you are trying to fwork out which of these will be quickest, I think that you be surprised how much faster the second one is - it does only the one memory allocation and a fill, whereas the first will doe a lot of memory allocations which can be seriously slow.


Time them: Use the Stopwatch class and try it. (Have a look at this: Counting lines in a string[^] - most of it shows timing the effects of different ways to do the same thing)
 
Share this answer
 
Comments
codeninja-C# 21-Nov-12 9:35am    
Thanks for your referenced link. Its really good and i will test my code using BFI
OriginalGriff 21-Nov-12 10:25am    
You're welcome!
1. I use MemoryStream unless I plan to save the file to the disk then go with FileStream. However, if you are going from stream to bytes MemoryStream is very helpful

2. I've completed an application that needed this an I didnt notice any speed reduction

XML
/// <summary>
      ///     Reads a Stream and outputs and return a byte array byte[]
      /// </summary>
      /// <param name="input"> Stream </param>
      /// <returns> byte[] </returns>
      public static byte[] ReadFully(Stream input)
      {
          using (var ms = new MemoryStream())
          {
              input.CopyTo(ms);
              return ms.ToArray();
          }
      }


This is the best soltion I found for my project after scouring the web for stream to byte array. 3 lines

An array is already split so how do you want it "split", give me some detail
 
Share this answer
 
v2
Comments
codeninja-C# 22-Nov-12 1:44am    
Hi,

I have to split the array based on certain length which is the actual length of the array and then i added into the list.
Is it possible to get the spitted array list from File Stream?

Thanks

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)



CodeProject, 20 Bay Street, 11th Floor Toronto, Ontario, Canada M5J 2N8 +1 (416) 849-8900