Click here to Skip to main content
14,452,204 members
Rate this:
Please Sign up or sign in to vote.
Hi,
I have several threads in a C application and i want them to write to a single log file without blocking
their run.
I probably need to do a worker thread which gets signaled by each thread when it wants to write to the file and then write the data which it got from the signaling thread

The order of the writing is important

How can i achieve this?

thanks!

dj

What I have tried:

Using Events but i cant understand how to send the writing data
Posted
Updated 28-Apr-19 7:51am
Comments
0x01AA 28-Apr-19 12:35pm
   
Use critical section

1 solution

Rate this:
Please Sign up or sign in to vote.

Solution 1

I wrote a library and utility to do exactly this many years ago. It was for event message logging. I ended up using a big buffer in shared memory to hold the logged message data and then I had a separate process that monitored the shared memory and wrote it to a file periodically. We used a mutex to control access to the buffer.

This meant the normal sequence of events to log an event was acquire the mutex, copy the message text to the first available buffer slot, and release the mutex. It was pretty fast too. We had dozens of processes running so it was worked pretty heavily. The file writing process waited until the buffer was almost full and then wrote the buffer to a file and cleared it. It also wrote the buffer at midnight when the day rolled over. This is because we kept one file for all the events of one day.

The buffer of messages was around 1MB and writing it to the file was slow. The writer had to open the log file, seek to the end of it, write all the data, and then close the file, all while holding the mutex. That took a surprisingly long amount of time which held up the worker processes and threads. To speed things up, I had it copy the entire buffer to a locally allocated buffer and then write the data from there. This took a little longer BUT it meant the mutex was held (and workers blocked) for less than a millisecond, just for the duration of the memory copy. The sequence of events for writing the message buffer was then allocate memory for the copy, acquire the mutex, copy the buffer, reset the buffer's counters, release the mutex, open the file and seek to its end, write the data, close the file, release the allocated memory.

It is important to note my system used multiple processes so I used a mutex to control access to the buffer. I also wrote this for a multiple threaded environment and for it I used a critical section to control access because it is much more efficient than a mutex. This is because a mutex is a kernel-level object that can work between multiple processes. A critical section is not and can work only between multiple threads of a single process. Since your system is a single process, a critical section would be best. The logic and implementation is essentially the same for both designs except the file writer is in a separate thread instead of a separate process and a critical section is used instead of a mutex for access control. Also, the buffer can be in local heap memory instead of shared memory but that is optional.
   
Comments
Afzaal Ahmad Zeeshan 28-Apr-19 15:52pm
   
Sounds interesting the way you put it! 5ed.

It would have been amazing to read the code, but I guess you won't have it.
Rick York 28-Apr-19 20:47pm
   
Thanks.

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)




CodeProject, 503-250 Ferrand Drive Toronto Ontario, M3C 3G8 Canada +1 416-849-8900 x 100