Click here to Skip to main content
12,692,778 members (33,467 online)
Rate this:
 
Please Sign up or sign in to vote.
See more: C#
Hi,

1. Which is better way to process the files (File Stream / Memory Stream)?
2. I think byte array decrease the process speed. is this true?

I need your valuable inputs to make the following code more efficient

Thanks in Advance
//split the array into multiple array and stored it in the list for analysis
 private void Split()
            {
                byte[] content = ReadFile(test.txt);
                List<string[]> record = new List<string[]>();
                int splitLength = 130;
                byte[] spiltResult = new string[130];
                for(int i=0; i<content.Length;i = i+spitLength)
                {
                    Array.Copy(content,i,spiltResult,0,splitLength);
                    record.Add(spiltResult);
                }
             content =nuul;
            }
//Returns the byte array of the given file
//Is this a right way to do? i need more efficient code
//Which is better File Stream/Memory Stream
 private byte[] ReadFile(string filePath)
            {
            FileStream fs = new FileStream(filePath, FileMode.Open, ileAccess.Read);
            int length = Convert.ToInt32(fs.Length);
            byte[] data = new byte[length];
            fs.Read(data, 0, length);
            fs.Close();   
            return data;
            }
           


Regards,
Sowraaj
Posted 21-Nov-12 3:45am
Updated 21-Nov-12 3:51am
v3
Comments
digimanus 21-Nov-12 9:00am
   
how to measure decrease in process speed?
Sowraaj 21-Nov-12 9:07am
   
Hi,

The byte array size is more than 10,00,000. i thought it probably occupies more memory space so that i come to the decision of worst process. What is the certain way to calculate the program process

Thanks for your responce
Sastry_kunapuli 21-Nov-12 10:02am
   
I would suggest to use file stream over Memory stream.typically both inherit from the Stream class,but I guess file stream is better equipped to handle files than memory stream.I did not understand your second question clearly
Sowraaj 21-Nov-12 23:38pm
   
Hi,

My second question is based on the array size. I converted the file stream into byte array. I thought when the array size is increasing, it will decrease the program process. Is it true?

and i also have another question which is based on array split
How can i store the splitted array into list from Memory Stream/ File Stream? (With out converting it into the byte array)
Sastry_kunapuli 26-Nov-12 13:09pm
   
Array is typically value based and is stored in the stack so any operations that you perform on the array increases the memory usage, and by decreasing program process you mean responsiveness of the program?
Mathlab 21-Nov-12 11:02am
   
3 line solution below
Rate this: bad
 
good
Please Sign up or sign in to vote.

Solution 1

Sowraaj,

The size of your file is just as important as the way that you read it in. When you're dealing with smaller files, you can read the whole file in and not worry too much about the method. However, when you're dealing with much larger files, say 10Mb and up, its a concern about what you're doing. It may be best to read them in incrementally. Say 100K at a time, process that bit of the file and then read in the next 100K disgarding the previous file contents from memory.

byte[] data = new byte[1024 * 100];
while(fs.Read(data, 0, length) > 0)
{ 
     Process File Contents Here
}




Hogan
  Permalink  
Comments
Sowraaj 21-Nov-12 9:25am
   
Hi,
I am processing only a 1MB file
Thanks
Rate this: bad
 
good
Please Sign up or sign in to vote.

Solution 2

If you are trying to fwork out which of these will be quickest, I think that you be surprised how much faster the second one is - it does only the one memory allocation and a fill, whereas the first will doe a lot of memory allocations which can be seriously slow.


Time them: Use the Stopwatch class and try it. (Have a look at this: Counting lines in a string[^] - most of it shows timing the effects of different ways to do the same thing)
  Permalink  
Comments
Sowraaj 21-Nov-12 9:35am
   
Thanks for your referenced link. Its really good and i will test my code using BFI
OriginalGriff 21-Nov-12 10:25am
   
You're welcome!
Rate this: bad
 
good
Please Sign up or sign in to vote.

Solution 3

1. I use MemoryStream unless I plan to save the file to the disk then go with FileStream. However, if you are going from stream to bytes MemoryStream is very helpful

2. I've completed an application that needed this an I didnt notice any speed reduction

/// <summary>
      ///     Reads a Stream and outputs and return a byte array byte[]
      /// </summary>
      /// <param name="input"> Stream </param>
      /// <returns> byte[] </returns>
      public static byte[] ReadFully(Stream input)
      {
          using (var ms = new MemoryStream())
          {
              input.CopyTo(ms);
              return ms.ToArray();
          }
      }


This is the best soltion I found for my project after scouring the web for stream to byte array. 3 lines

An array is already split so how do you want it "split", give me some detail
  Permalink  
v2
Comments
Sowraaj 22-Nov-12 1:44am
   
Hi,

I have to split the array based on certain length which is the actual length of the array and then i added into the list.
Is it possible to get the spitted array list from File Stream?

Thanks

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

  Print Answers RSS
Top Experts
Last 24hrsThis month


Advertise | Privacy | Mobile
Web02 | 2.8.170118.1 | Last Updated 21 Nov 2012
Copyright © CodeProject, 1999-2017
All Rights Reserved. Terms of Service
Layout: fixed | fluid

CodeProject, 503-250 Ferrand Drive Toronto Ontario, M3C 3G8 Canada +1 416-849-8900 x 100