How to decide ideal number of threads ?

There is no easy correct answer to the question.


It really depends on if/whether your threads are ever going to be waiting for i/o (disk access, network, etc.) and how long those waits will be versus the cost of switching threads.

If your processor core is only ever going to be waiting on data from RAM, then it's unlikely the processor can switch threads faster than that.

If on the other hand, the thread is going to be waiting on data from some network attached database, then the system might be able to switch threads and get something useful done while waiting for it. In that case it might be useful to have twice as many threads as cores.

The other thing to take into account, is what (if anything) else is going to be running in the system, and is the OS going to be a lot of other things? And how does the OS assign cores? (In which case it might turn out that 1 less thread than the number of cores might be best.)

The best thing to do is try various numbers of threads and see what gives you the best performance. I'd try a couple of different test cases. If you have N cores, I'd try 1-thread, N-1 threads, N threads, N x 2 threads -- and see what gave you the best performance consistently. If N threads and N x 2 threads give similarly good results, try N x 1.5 and N x 3.