Click here to Skip to main content
15,880,725 members
Please Sign up or sign in to vote.
0.00/5 (No votes)
I am currently working on a program that receives UDP packets of size 1460 bytes per packet and continuously streams the bytes to the hard drive of my PC. Currently, my program can only receive and stream 84 packets before data loss occurs. I need my program to be realtime with a fast sampling rate of 20 kHz. I need to be able to receive much more packets of data without loss.

My code connects to the client and waits for data to come in. When data comes in, the program receives the data, streams to the hard drive, then waits for more data. I think that I am not streaming fast enough, so I fail to receive the next packet of data that gets sent in.

Here is some code:

C#
if (firsttimeflag == false) // Streaming each value to the hard disk 
   {
    using (BinaryWriter b = new BinaryWriter(File.Open(@"F:\file.bin", FileMode.Create)))
       {
           //Use foreach and write all values.
           foreach (int i in datain)
           {
             b.Write(i); // Writes a four byte signed integer.
           }
       }
             firsttimeflag = true;
     }
 else
     {
      using (BinaryWriter b = new BinaryWriter(File.Open(@"F:\file.bin", FileMode.Append)))
          {
            foreach (int i in datain)
              {
                b.Write(i);
              }
          }
      }


Where "datain" is the 1460 byte array which is converted to a string beforehand.

Any help would be appreciated. Where is the inefficiency of my code?

Thanks,
GabeA
Posted

Why are you messing about with the data in this way? You receive a stream of bytes from the network so all you need to do is write that stream (whatever length you receive) to the disk. Don't try converting to a string (what does that achieve?) and don't write four bytes as an int.
 
Share this answer
 
Comments
GabeA 10-Jan-12 12:03pm    
The reason I was converting to a string was because I am receiving a 1460 byte array, and the first few bytes are used for indentifiers and other header that I am receiving from the client. Using the System.Text.Encoding.UTF8.GetString(data, index, count) command is the only way that I know how to partition the incoming data. What is a way around this?
Richard MacCutchan 10-Jan-12 12:24pm    
Why do you want to partition it in the first place? If you want efficiency then read from the network - write to the disk. Leave the data exactly as it is, you can process it into its constituent parts later. If you need to identify each block of data then write a prefix word to the disk that gives the length, in front of each block.
Sergey Alexandrovich Kryukov 10-Jan-12 12:11pm    
Agree, a 5. My congratulations with another MVP!
--SA
Richard MacCutchan 10-Jan-12 12:20pm    
And to you; I can't believe this is only your first MVP award.
I few things which should help on the speed front...

Open the file and keep it open, and just write data to it as it comes in, and then close the file when the program exits. Open and closing files for each packet is just overhead which can easily removed.

Why are you writing each 'int' by itself to the file, why not just send the whole 'datain' buffer to be written to disk? It removes the loop(s) which are just more useless overhead.

Also you may want to check out unbuffered file writes, as buffering done by C# and Windows will mean your program will 'pause' as Windows/C# realises the buffer is full and should be written to disk... You shall end up with Java-ish GC pauses with buffering if your are unlucky.
 
Share this answer
 
Comments
GabeA 10-Jan-12 12:19pm    
I understand that my code is inefficient, but I am not sure exactly how to send the whole datain buffer to be written to the disk instead of breaking up the data into individual ints.
Richard MacCutchan 10-Jan-12 12:26pm    
Use the MSDN documentation to help you. As you can see in this page you can write an array of bytes directly to your BinaryWriter.

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)



CodeProject, 20 Bay Street, 11th Floor Toronto, Ontario, Canada M5J 2N8 +1 (416) 849-8900