Click here to Skip to main content
15,844,903 members
Please Sign up or sign in to vote.
4.00/5 (1 vote)
See more:
I’m working towards changing a vb.net program from standard to a multi threaded model.

It runs as a windows service and has no GUI and performs a number of complex reporting and file generation functions on demand for the multiple users simultaneously when requested by the user.
For me to see what the program is doing it’s essential to have good logging which currently logs to a text file when “trace” is turned on. It app captures a few hundred lines in the log for each run.

I see some issues with using a single text file under multi threading.
I thought a database would make it easy to query and follow each thread despite the processing chronology of other threads in-between. I tried MySQL database for storing logs, the problem was logging slowed the app from 3 seconds for a standard run to 34 seconds. That’s a big performance hit.

Now I’m back to the idea of using multiple text files but then uploading it to MySQL afterwards and then dealing with logical presentation later.

I read about people using the below method
TextWriter tw = TextWriter.Synchronized(File.AppendText(filePath));

I’m thinking would that slow it down juggling between threads?
At the moment I’m leaning towards the idea of generating a text file name for each thread based on the GUID assigned for that thread and then later uploading the contents of all the disparate files into MySQL afterwards.

The thinking behind this is that if each thread has its own StreamWriter running it wont be juggling concurrency with any of the other threads wanting to write to a text file.

The objective is keep everything running as fast as possible the whole way.
Does anyone have any input or had experience around this with regard to speed of logging in a multi threading environment?
Posted

I appreciate you've probably already got a codebase, but possibly check out log4net[^]

I use this this very successfully in a high volume, mutil-threaded environment.

http://stackoverflow.com/questions/50213/are-there-any-issues-with-using-log4net-in-a-multi-threaded-environment[^]

http://stackoverflow.com/questions/1519211/multithread-safe-logging[^]

If you're not familiar with it, I suggest you give it a go. No point reinventing the wheel :)
 
Share this answer
 
If you are going to hold a stream for each thread anyway, why not use a MemoryStream? Why back it with a text file unless you expect the app to crash out?
 
Share this answer
 
Comments
Dave Kreskowiak 31-Mar-11 12:19pm    
One downside of this would be if the logs generated are extensive. You could run the machine out of RAM. Yes, I speak from experience. no, I didn't come up with that idea - I just witnessed the effects! ;)
OriginalGriff 31-Mar-11 15:03pm    
This is true, but since the OP said that a standard run took three seconds, I wasn't too worried!
If you need logging, you better use logging, not home-based techniques.

See my post in response to these Questions:

This is how to make a custom logger (to log to text, to UI or whatever):
MsBuild OutPut to the TextBox on the fly in Windows Application[^]

This is how to log to the System log, but with structure:
How to create event log under a folder[^]
I offered a whole complete wrapper code here.

—SA
 
Share this answer
 

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)



CodeProject, 20 Bay Street, 11th Floor Toronto, Ontario, Canada M5J 2N8 +1 (416) 849-8900