Click here to Skip to main content
12,999,417 members (56,047 online)
Rate this:
Please Sign up or sign in to vote.
See more:
Hello There .

I have multithreaded TCP IP server which must send 100 messages per second to its clients.
I am testing it on 500 clients and the size of each message around 2kb.

I developed my server to be a multithreaded application using SMART Thread Pool.

The multithreading is working fine. I am using socket .beginsend method to send messages to clients.

While testing I face a delay problem:
If we said 100 message per second to 500 clients this means 50000 message per second this takes 18 seconds to be sent to clients.

I test to see if the problem is with my threading application execution of threads but I found that when I comment the beginsend line I find that the execution does not need more than 2 seconds reaching the sending method.

Is there is a way to speed up the execution of beginsend?
Is this behavior correct ?
Posted 4-Nov-10 3:20am
Updated 8-Nov-10 21:49pm
Dalek Dave433.3K
Dalek Dave 9-Nov-10 3:49am
Edited for Grammar, Spelling and Syntax.
Eddy Vluggen 9-Nov-10 6:07am
How are you assigning this BeginSend to the SMART-threadpool? Sounds like you're running into the maximum number of threads from the threadpool, where the application has to wait until there's a new free thread available. Also, it does cost time to start a thread, and starting threads cannot be speed up.
hazem12 9-Nov-10 7:49am
thanks for your comment.
First of all I was sending each message to an client as a job assigned to thread pool and I made IWorkItemsGroup for each client to make sure that all messages sent to client in order but on about 100 client with the rate less than 100 message per second I faced problem that the CPU always more that 95 % I made some search about that and I found that the sending job to one client is fast job and it maybe cost less than assigning the job to thread pool so. I modified the code to make the job to be sending to list of clients and the number in the list depends on a number I assigned in the program configuration so if we have 100 clients and the number I assigned is 5 so each list we have 25 clients. But also on this way I faced a little problem on 500 clients with assigned number about 32 so each list has 16 clients we have the problem I mentioned and the CPU be always more than 95% but if I modified the number to 500 I still have delay as I mentioned in my problem but the CPU reduced to 60% also even if I have 1000 client with 500 number assigning .but the delay problem still there . and I set the maximum number in the thread pool to high number about 500 thread and the all threads application not more 170 thread according to task manager.
Rate this: bad
Please Sign up or sign in to vote.

Solution 1

This is very simplified - I ignore packet overhead, acks, response turnround, switch backplane capacity, and unicode.

2KB = 2096 * 8 bits.
100 messages per second = 100 * 2096 * 8.
500 clients = 500 * 100 * 2096 * 8.
= 838,400,000 bits per second.

Are you using a gigabit network? Because if you aren't there is your bottleneck, right there.
Dave Kreskowiak 6-Jan-15 20:16pm
Nevermind. I didn't look at the posting dates.

I hate that CP put's a "touched" message at the top of the queue.
Rate this: bad
Please Sign up or sign in to vote.

Solution 3

I believe that even if you have enough bandwidth to cope with the amount of data you want to send, there's too much overhead and latency with sending that many packets. When sending over TCP you need to send the packet which will have a small delay as it gets sent onto the actual network (regardless of the time it takes to actually travel to the other computer) and then you need to wait for an acknowledgement that the client has received the packet before continuing or if no ack is received, send the packet again.

Also with using packet sizes of 2kb you're packets may be being fragmented as they don't actually fit within one frame of data, which will obviously take a little longer to send. Also if you have a small maximum packet size then the messages will actually be split into separate TCP packets which is going to incur quite a lot more overhead.

Page about MTU[^]

The problem your having may be more to do with the limitations of the network rather than the performance of the code. What you might like to try is sending the same amount of much smaller messages and see how it copes.
Rate this: bad
Please Sign up or sign in to vote.

Solution 2

Yes I am using Gigabit network. I forget to mention that when I comment the sending statement also the CPU usage reduced to half of usage when the statement uncommented
Rate this: bad
Please Sign up or sign in to vote.

Solution 4

Hello i tried to send small message there is no big change the time reduced to 2 minutes and this is not good .
what i can do to reduce latnecy .
Rate this: bad
Please Sign up or sign in to vote.

Solution 5

Hello agian,
I cant still find an answer to my question here .
i need to know if you please . Am i reach the TCP/IP limit ? in my case or still the TCP/IP can affored more than that ?
how I can solve my problem?
Dave Kreskowiak 22-Aug-13 23:51pm
Dude. Go back and reread these answers. If you're using a Gigabit Ethernet NIC, YOU'RE EXCEEDING THE LIMITS OF YOUR NETWORK HARDWARE!

There may be hardware solutions that MAY help you do this and, depending on your network infrastructure, may also be hindering you.

First, NIC teaming. Where multiple NICs in your server (you ARE running this on Windows Server, correct?) are setup in a team configuration to increase available bandwidth. The switch your server is connected to must also support teaming!

Second, get a faster NIC. Just based on the simple math of the number of messages you're sending you are exceeding the limits of Gigabit Ethernet. If you want to send that kind of data you'll need a faster NIC. The problem is that there are not too many 10Gb NICs out there and your switch hardware will also have to be upgraded to handle this speed.

Server cluster. Redesign your app to run on a cluster of servers, spreading the traffic load out across multiple servers. What will limit you here will be your network infrastructure switching capacity.

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

  Print Answers RSS
Top Experts
Last 24hrsThis month

Advertise | Privacy | Mobile
Web01 | 2.8.170624.1 | Last Updated 6 Jan 2015
Copyright © CodeProject, 1999-2017
All Rights Reserved. Terms of Service
Layout: fixed | fluid

CodeProject, 503-250 Ferrand Drive Toronto Ontario, M3C 3G8 Canada +1 416-849-8900 x 100