Multithreading in calling a same function VS2010 _beginthread

Question

1.00/5 (2 votes)

See more:

VS2010

, +

Hello all,

I have function in C using VS2010 Prof, which reads 1 data file(from 1000 data files), do some analysis and writes some data files in another directory, then continues reading and writing to the end. generally my function is myfunc(int start_index,int end_index,char *read_file_directory). To improve the runtimes I created 5 threads using _beginthread each of which calls myfunc as follows:
thread 1: myfunc(1,200,read_directory)
thread 1: myfunc(201,400,read_directory)
thread 1: myfunc(401,600,read_directory)
thread 1: myfunc(6011,800,read_directory)
thread 1: myfunc(801,1000,read_directory)
myfunc works very well over 1000 files and I checked it with Visual Leak Detector 2.3, however it shows no memory leak. As soon as using multithreading, my program gives me access violation. I wrote some small functions and call them in myfunc also I am using CLAPACK (32bit release version) and fftw3f. I am totally confused, what is the reason? how I can fix it any help is appreciated.

Posted 31-Dec-14 10:42am

rravanfar

Add a Solution

Comments

Sergey Alexandrovich Kryukov 31-Dec-14 18:34pm

You did not provide any information on having access violation. Perhaps you share something between threads without proper synchronization (but why?); I just don't know.

Besides, having 5 threads for such things looks pointless. I don't believe you have 6 or more CPU cores which could justify such things. You do need to use multithreading, but adding just one extra thread would usually be about optimal. Besides, you could use TPL instead explicit threads.

—SA

Kornfeld Eliyahu Peter 1-Jan-15 14:06pm

Access violation of what? The files? How do you open those files?

KarstenK 2-Jan-15 6:21am

Without code it is only guessing, but the access violation shows a serious bug. Using a global handle or memory?

Tip: check that every thread is clean in memory managment!!!

rravanfar 2-Jan-15 11:16am

friends, I put the main part of my code, myfunc is main function called in threadfunc. Please let me know if there is anything wrong in creating the threads and dividing the job between them. One point comes to my mind; in myfunc I allocate many arrays (totally around 100M)and another function which is called inside myfunc allocates another around 100M(including many arrays and fftw plans and freeing them during the code)memory and then frees it. So, I was thinking I need more memory if I want to use 4 to 7 threads(my CPU is corei7) so I changed the project setting to access long address (all external libraries and VS2010 are 32bits but my windows is 64bit).

rravanfar 2-Jan-15 11:23am

Another point, the only common file that myfunc has to open it at the beginning is the:
WaitForSingleObject(ghMutex, INFINITE);
Param=ReadTestPara(base_path);
ReleaseMutex(ghMutex);
so I isolated readtestpara using the above code.

1 solution

Add a Solution

Add your solution here

Treat my content as plain text, not as HTML

Preview 0

…

Existing Members

Sign in to your account

...or Join us

Download, Vote, Comment, Publish.

Your Email
Password
Forgot your password?

Your Email
This email is in use. Do you need your password?
Optional Password

I have read and agree to the Terms of Service and Privacy Policy
Please subscribe me to the CodeProject newsletters

When answering a question please:

Read the question carefully.
Understand that English isn't everyone's first language so be lenient of bad spelling and grammar.
If a question is poorly phrased then either ask for clarification, ignore it, or edit the question and fix the problem. Insults are not welcome.
Don't tell someone to read the manual. Chances are they have and don't get it. Provide an answer or move on to the next question.

Let's work to help developers, not make them feel stupid.

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

George Jonsson · Answer 1 · 2015-01-02T05:49:00

There is often not very efficient to try to read files from the hard drive in parallel.
What you can do instead is to read the the files from disk sequential in a loop into memory and then start a thread that analysis the data and writes the result back to a file. When the thread method is done executing, it exits.

You can try to use a CriticalSection[^] in order to avoid simultanious disk operation and see if you gain or lose in execution time or if you get access errors without the CriticalSection. (Which you shouldn't if you write to disk using different file names.

If this is a good approach or not depends on the size of the files and how much time the analysis takes.
One risk with this approach is that you might end up with a lot of threads if the analysis time is long compared to the time to read the file.