Sending Large Files in C# using TCP Client/Server [modified] - C# Discussion Boards

Not Working:
Error:Object reference not set to an instance of an object.
I am not able to use FindControl[] in windows applications using c# .net.
So please does any one will provide my the suitable code.

hi, my program is open all day and access to sql frequently. So, is it good practice to keep sql connection open and close it when the form closes? OR should be closed after each access?.

It really depends. Do you plan to have only one user connect to the database? Bear in mind that a database connection is a finite resource, and that it has implications on server maintenance, so you need to review the architecture of your whole system.

In general, it's better to release the connection and reacquire it later on from a connection pool (if the database and driver supports pooling) because this is less of a hit than creating the connection anew each time.

"WPF has many lovers. It's a veritable porn star!" - Josh Smith

As Braveheart once said, "You can take our freedom but you'll never take our Hobnobs!" - Martin Hughes.

My blog | My articles | MoXAML PowerToys | Onyx

my database and the program are on the same computer.There is no pool, database and the program is used by only one computer one person. Recently, every minute or so, the program accesses sql.What would your suggestion be? keeping it open?

teknolog123 wrote:
There is no pool

If you have SQL Server and ADO.NET then you have a connection pool. Read a little about it and you will understand, http://msdn.microsoft.com/en-us/library/8xx3tyca(VS.80).aspx[^]

I know the language. I've read a book. - _Madmatt

To be honest, I would close the connections as soon as I finish with them. This frees up valuable resources (including memory).

"WPF has many lovers. It's a veritable porn star!" - Josh Smith

As Braveheart once said, "You can take our freedom but you'll never take our Hobnobs!" - Martin Hughes.

My blog | My articles | MoXAML PowerToys | Onyx

Open and close for each operation.

Hi All

I've been struggling for a while now trying to solve this issue of sending larger files across TCP client/server program in c#. Been Googling for days with partial solutions to the problem. Every time I think that I have found a solution, something bombs out. Frown | :(

Brief Explanation:
The Client sends a requests, the Server performs an operation on a file, and then sends the file to Client, which then receives this file.

For smaller files, the normal method of changing file into byte array works only for smaller files.
I have also tried converting the file to base64string and then send it that way, but also doesn't work.

Could someone please help me solve the problem of sending larger files using TCP?

These are the 3 main techniques that seemed promising:
i.

inFile = new System.IO.FileStream(theFileName, System.IO.FileMode.Open, System.IO.FileAccess.Read);
   binaryData = new Byte[inFile.Length];
   long bytesRead = inFile.Read(binaryData, 0, (int)inFile.Length);
   inFile.Close();
   //Convert the binary input into Base64 UUEncoded output.
   string base64String;
   base64String = System.Convert.ToBase64String(binaryData, 0, binaryData.Length);
   byte[] buffer = encoder.GetBytes(base64String);
   serverStream = tcpClient.GetStream();
   serverStream.Write(buffer, 0, buffer.Length);
   serverStream.Flush();

ii.

byte[] fileNameByte = Encoding.ASCII.GetBytes(theFileName);
      if (fileNameByte.Length >; 850 * 1024)
      {
         Console.Out.WriteLine("File is too big");
         return;
      }
      byte[] fileData = File.ReadAllBytes(theFileName);
      byte[] clientData = new byte[4 + fileNameByte.Length + fileData.Length];
      byte[] fileNameLen = BitConverter.GetBytes(fileNameByte.Length);
      fileNameLen.CopyTo(clientData, 0);
      fileNameByte.CopyTo(clientData, 4);
      fileData.CopyTo(clientData, 4 + fileNameByte.Length);
      tcpClient.Client.Send(clientData);

iii.

const int BufferSize = 1024;
     byte[] SendingBuffer = null;
     FileStream Fs = new FileStream(theFileName, FileMode.Open, FileAccess.Read);
     int NoOfPackets = Convert.ToInt32(Math.Ceiling(Convert.ToDouble(Fs.Length)/Convert.ToDouble(BufferSize)));
     int TotalLength = (int)Fs.Length, CurrentPacketLength, counter = 0;
     for (int i = 0; i < NoOfPackets; i++)
     {
        if (TotalLength > BufferSize)
        {
           CurrentPacketLength = BufferSize;
           TotalLength = TotalLength - CurrentPacketLength;
        }
        else
        {
           CurrentPacketLength = TotalLength;
           SendingBuffer = new byte[CurrentPacketLength];
           Fs.Read(SendingBuffer, 0, CurrentPacketLength);
           serverStream.Write(SendingBuffer, 0, (int)SendingBuffer.Length);
        }
        Fs.Close();
     }

__________________________________

I am not sure if the problem might lie on the client side when receiving the file, so I have listed that too (in the same order as above):
i.

//get the file
     byte[] getFile = new byte[4096];
     bytesRead = 0;      
     bytesRead = clientStream.Read(message, 0, 4096);
     string base64String = encoder.GetString(message, 0, bytesRead);
     byte[] binaryData = Convert.FromBase64String(base64String);
     FileStream fs = new FileStream(theFileName, FileMode.Create, FileAccess.ReadWrite);
     BinaryWriter bw = new BinaryWriter(fs);
     bw.Write(binaryData);
     bw.Close();

ii.

byte[] clientData = new byte[1024 * 22000];
     int receivedBytesLen = m_tcpclient.Client.Receive(clientData);
     int fileNameLen = BitConverter.ToInt32(clientData, 0);
     string fileName = Encoding.ASCII.GetString(clientData, 4, fileNameLen);
     BinaryWriter bWrite = new BinaryWriter(File.Open(theFileName, FileMode.Append)); ;
     bWrite.Write(clientData, 4 + fileNameLen, receivedBytesLen - 4 - fileNameLen);

iii.

string SaveFileName = string.Empty;
     SaveFileName = theFileName;
     int RecBytes;
     const int BufferSize = 1024;
     byte[] RecData = new byte[BufferSize];
     if (SaveFileName != string.Empty)
     {
        int totalrecbytes = 0;
        FileStream Fs = new FileStream(SaveFileName, FileMode.OpenOrCreate, FileAccess.Write);
        while ((RecBytes = m_networkstream.Read(RecData, 0, RecData.Length)) > 0)
        {
           Fs.Write(RecData, 0, RecBytes);
           totalrecbytes += RecBytes;
        }
        Fs.Close();
     }

________________________

Thanks for the help in advance! Laugh | :laugh:

modified on Tuesday, April 6, 2010 8:39 AM

I dont think you've made it clear exactely what your issue is with large files

The general approach is to

1. read n buffers of size x from file
2. transmit the 1..n buffers over the wire
3. re-assemble them in the correct order

obviously a huge buffer size (x) isnt efficient - have you tried sending n smaller buffers with a sequence tag and then reassembling them ?

'g'

To be honest, I am not sure what the problem is.
I think that I am not sending all n buffers, as you suggested because my file when it is recieved on the client side is always smaller than the original, hence I must not be sending it correctly.
Not quite sure which one of the techniques I used was best, or even whether I implememnted them correctly.
I am pretty much open to all suggestions as I am quite new to c#.

Thanks for the message! Appreciated.

I found an interesting link here on CP about BITS - may be what you need

Forms Designer Friendly Background Intelligent Transfer Service (BITS) wrapper[^]

other links that may help :-

http://stackoverflow.com/questions/471726/good-way-to-send-a-large-file-over-a-network-in-c[^]

http://www.csharphelp.com/2006/10/client-server-simple-file-sharing-and-file-transfer/[^]

'g'

Hi
Thanks for the links...have gone through some one of them already from a previous Google search.
One of the other links contains good explanations, and more links, which I checked too.

Still finding it difficult to recreate the same file on the other end. Frown | :(

Thanks

If you have a method for sending files that are small them it should work for larger files, it will just take longer to transmit the file. It would help if you specified the file sizes for what you are calling small and large files.

All of the methods for sending data via TCP load/convert the data to byte[] and send that so avoid converting the file to Base64 as this just increases the amount of information that has to be sent over the network, which then takes longer to transmit.

The small file was 11kb, and the larger file was 1.5MB (not that large).

I initially used the same method as sending text (via byte streaming) to send the files, but as the files I tested got larger, it wouldnt work any longer, which has lead me to testing different methods of sending files.

I've sent files up to 5MB using something similar to;

Byte[] data = System.Text.Encoding.ASCII.GetBytes(message);         

TcpClient client = new TcpClient(server, port);
NetworkStream stream = client.GetStream();
stream.Write(data, 0, data.Length);
client.Close();

It did take a while to transfer though of the order of 3-5 minutes.

For receiving the file look at the TcpListener Class[^]

Surely that would then mean converting to base64 first?
Thought that this might not be ideal or practical as files become larger.
But I will give it a go...

Thanks

You don't need to convert to Base64 first, just load your file into a byte[] and send it, then when receiving don't do any byte to string conversions either, just save the received bytes to a byte[] that keeps growing or save the bytes to a file on disk.

Sorry for being such a noob here...this is the code I used:

System.IO.FileStream inFile = new System.IO.FileStream(theCompressedFile, System.IO.FileMode.Open, System.IO.FileAccess.Read);
byte[] binaryData = new Byte[inFile.Length];
long bytesRead = inFile.Read(binaryData, 0, (int)inFile.Length);
inFile.Close();

Byte[] data = System.Text.Encoding.ASCII.GetBytes(binaryData);
serverStream.Write(data, 0, data.Length); client.Close();

"System.Text.Encoding.ASCII.GetBytes" won't accept binaryData (byte[]) as an input.

ClientSide:

byte[] message = new byte[4096];
int bytesRead = 0;
bytesRead = clientStream.Read(message, 0, 4096);
file = encoder.GetString(message, 0, bytesRead);

FileStream fs = new FileStream(theFileName, FileMode.Create, FileAccess.ReadWrite);
BinaryWriter bw = new BinaryWriter(fs);
bw.Write(file);
bw.Close();

Thanks for the help...and sorry if this is annoying. Frown | :(

No problem change the send to;

serverStream.Write(binaryData, 0, binaryData.Length);

drop the data byte[], then try;

FileStream fs = new FileStream(theFileName, FileMode.Create, FileAccess.ReadWrite);
BinaryWriter bw = new BinaryWriter(fs);

byte[] message = new byte[4096];
int totalBytesRead = 0;
int bytesRead = clientStream.Read(message, totalBytesRead, 4096);

while (bytesRead  > 0)
{
    bw.Write(message, totalBytesRead, bytesRead);
    message = new byte[4096];
    totalBytesRead += bytesRead;
    bytesRead = clientStream.Read(message, totalBytesRead, 4096);
}
       
bw.Close();

on the receive.

modified on Tuesday, April 6, 2010 8:54 AM

Woah, cool...that works.
For some reason, the files I test are slightly larger than the original, and won't open on the received client side due to corruption. Frown | :(

On days like these, just want to drop the PC and never touch it again...but I can't, I can't get away from the use of a PC. Smile | :)

Thanks Again for all your efforts, I really appreciate it!

There shouldn't be any difference in the file size.

Also on the receive side you could drop the BinaryWriter and just write the bytes received direct to the FileStream.

FileStream fs = new FileStream(theFileName, FileMode.Create, FileAccess.Write);

byte[] message = new byte[4096];
int totalBytesRead = 0;
int bytesRead = clientStream.Read(message, totalBytesRead, 4096);
while (bytesRead > 0)
{
   fs.Write(message, totalBytesRead, bytesRead);
   message = new byte[4096];
   totalBytesRead += bytesRead;
   bytesRead = clientStream.Read(message, totalBytesRead, 4096);
}

fs.Flush();
fs.Close();

With this new method, an exception is thrown when reading from the clientStream
i.e. once in the while loop, it cannot read from clientStream
Exception: "Specified argument was out of the range of valid values. Parameter name: size"

Sorry late night Roll eyes | :rolleyes:

;

FileStream fs = new FileStream(theFileName, FileMode.Create, FileAccess.Write);

byte[] message = new byte[4096];
int bytesReceived = 0;
int bytesRead = clientStream.Read(message, 0, 4096);

while (bytesRead > 0)
{
    fs.Write(message, 0, bytesRead);

    bytesReceived += bytesRead;
    message = new byte[4096];
    bytesRead = clientStream.Read(message, 0, 4096);
}

fs.Close();

Hi There

Was a long day for me yesterday too...I got it sorted just before leaving work yesterday.
Thanks again for all the help, I really appreciate it.

Enjoy the rest of ur day! Thumbs Up | :thumbsup:

Sorry, I'm not going to read such amount of unformatted code. Please use PRE tags to preserve code formatting.

Smile | :)

Luc Pattyn [Forum Guidelines] [Why QA sucks] [My Articles]

Getting an article published on CodeProject should be easier and faster for Bronze and Silver authors.