Click here to Skip to main content
Rate this: bad
good
Please Sign up or sign in to vote.
Hi,
 
I am developing an application where application suppose to process the data of records (ranges from 1 - 500000). For each block i am creating threads,
 
If i am creating /destructing threads for at least 11k times, then .... is it the overhead ??
 

Thanks in advance.
-Prafulla
 

 
UPDATE :
 
I tried with creating thread at once. But it is adversaly affecting the performance. It is taking more time than creating threads at each time.
 
Is ther any thread specific memory related issue ???
 
NEED HELP
Posted 20-Jun-11 22:39pm
Edited 22-Jun-11 22:16pm
v2
Rate this: bad
good
Please Sign up or sign in to vote.

Solution 1

Probably yes. You should arrange a little test to verify execution times. You may also have a look at Thread Pool Pattern[^].
  Permalink  
Comments
Albert Holguin at 21-Jun-11 9:19am
   
my 5, thread creation overhead can be significant, if there's a case where you have to create multiple threads, you're better off creating them once and tasking them.
Joan Murt at 21-Jun-11 9:25am
   
5ed!
SafarTimura at 22-Jun-11 4:21am
   
Agree 5
 
I didn't know about design patterns but they are obviously very very useful.
Rate this: bad
good
Please Sign up or sign in to vote.

Solution 2

Yes creating and destroying threads do have an overhead, perhaps the better option is to have one manager thread and then have worker thread to which you assign the work to from the manager thread.
 
hence you only create threads at the start of the program and have the wait on a message telling them which records to process.
Once done they send a message to the manager thread telling it they are done and want the next set of records to process.
  Permalink  
Comments
KarstenK at 21-Jun-11 9:59am
   
He needs a global queue which is thread safe to distribute the queued jobs.
The count of threads should be balanced which count is optimal to perform. Timeouts, parallel tasks and dead locking are problems.
SafarTimura at 21-Jun-11 10:06am
   
Yes that is a fair point. The gobal queue is a must.
 
I was assuming that he must have thought about timeouts, parallel tasks and dead locking if he is creating 11K worth of threads already and wants the task to be completed successfully.
KarstenK at 21-Jun-11 10:12am
   
my tip is that 10 threads are enough, so 100 wont be much faster if he isnt populating a server farm.
 
Threading isnt the solution, but intelligent and clever code ;-)
SAKryukov at 21-Jun-11 16:35pm
   
Agree, my 5.
I added my 2 cents in my answer.
--SA
Rate this: bad
good
Please Sign up or sign in to vote.

Solution 3

Very unreasonable overhead. You should re-think your design totally. There are could be different reasons for using threads. Do you need to more fully use your CPUs/cores (but then you hardly need more CPUs than cores) or thread represent logical entities of your application (such as in the famous problem of dining philosophers).
 
It is very likely that you can use one of a fixed small number of threads. You can also use the Thread Pool pattern, see http://en.wikipedia.org/wiki/Thread_pool_pattern[^].
 
—SA
  Permalink  
Comments
SafarTimura at 22-Jun-11 4:17am
   
Agree 5 :)
 
I didn't know about design patterns but they are obviously very very useful.
SAKryukov at 22-Jun-11 4:18am
   
Thank you, Safar.
--SA

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

  Print Answers RSS
0 OriginalGriff 7,903
1 Sergey Alexandrovich Kryukov 7,192
2 DamithSL 5,604
3 Manas Bhardwaj 4,986
4 Maciej Los 4,820


Advertise | Privacy | Mobile
Web02 | 2.8.1411023.1 | Last Updated 23 Jun 2011
Copyright © CodeProject, 1999-2014
All Rights Reserved. Terms of Service
Layout: fixed | fluid

CodeProject, 503-250 Ferrand Drive Toronto Ontario, M3C 3G8 Canada +1 416-849-8900 x 100