|Multi threading is anything but boring.
A few days ago, I was asked to improve the performance of a program, which reads a FOX Pro table, generates SQL
INSERT commands, and executes them against a database.
The way to do it was clear: read in one thread, write on another. To spice things up, since inserting is much slower than reading (all that index updates, you know), three writer threads were matched to the reader.
The writers' thread procedure looked like this (pseudo-code):
DWORD WINAPI ThreadProc(LPVOID pX)
// Cast passed pointer to what it really is.
// Initialize COM
// Generate message pump
// Connect to database
case WM_COMMAND: // Cast LPARAM to pointer to a Job, execute that job.
done = true;
Tested it with 16K records: runs like the wind, results are fine.
64K records: still OK.
80K records: some start to get lost.
I've use ::PostThreadMessage() for quite some time: the caller allocates a Job, posts it to the queue, the worker thread pops it from the queue, executes it, and deletes it. All synchronization tasks are taken care of by the OS, which minimizes thread switches.
Well, you always have to read the fine print. And the fine print for ::PostThreadMessage() is that the queue size is 10,000 messages. That's a lot, if you have a few dozen long-running jobs (in the past, for me, that was the usual).
So, how could I handle it?
My first idea was the use of a 'thermostat'. An integer was interlock-incremented whenever a message was posted, and interlock-decremented whenever a job was executed, thus holding the count of messages in the queue.
The manager thread would test the queue size: if longer than 900 messages, it would
Sleep for 50 milliseconds: if, after that, there were less than 1000 messages, it would try to post one (in a loop); if, after 5 tries the message was not posted yet, the job would be executed on the calling thread.
I didn't like it: whole lot of interlocking going on, and Sleep is by far not my favorite API.
Well, the bad news was as expected: performance went down by 30%, but still it was three times faster than the original program.
The good news: 80K records, and none got lost. 150K, and none got lost!
So I tried for 400K (the program should be able to send about a million): well, it got stuck after 215K. Playing with the thermostat settings helped a bit, but no cigar.
After a couple of hours, I gave up. Added
std::queue to the mix, managed synchronization with a critical section (which protected this queue), and performance went back to good. Private bytes and CPU usage were totally flat, up to 400K records.
The program was ripe for the QA department.
What did I learn?
::PostThreadMessage() is a nice tool, but it has some limitations. Its usage: when you have just a few 'big' jobs, and in a program which runs, does its thing and exits. For long-running programs, or programs which send to the worker threads a lot of small jobs, you're better off rolling your own queue.
And yet, multithreading, when you know how to do it, can be a great way to make an impression, and to become the star of your company parties.