Centralized Asynchronous Logging

If you've been working long enough in application development like me, I'm sure you are aware of the benefits of incorporating a good logging mechanism within your application.

Recently, I was designing an SOA based enterprise platform. There were a number of services and applications that required some serious logging and honestly, I wouldn't have given it much thought and just included some third party component such as log4net or Microsoft Enterprise Application Logging Block that would log data to a text file within each application or service.

But I had a second thought when re-factoring my architecture and being a firm believer of consolidating and centralizing application development and anything related that perfects the reuse of code, process or data. I thought there must be a better way to implement logging?

So I made a list of woes I had experienced from previous logging implementations. The most significant thing that hit me was the difficulty involved in accessing logging information when things went wrong. You had to locate the application, the server then scans through a huge massive log file and you had to be the one who has access to this file. That meant a significant down time even if the solution was a tiny change in the config.

So the most possible solution was centralized logging, but wait... what about performance? Logging to a flat file surely meant a hit on performance. So let's have centralized asynchronous logging is what I came up with.

Centralized Asynchronous Logging

I used Enterprise Application Logging Distributed Services to implement Centralized Asynchronous Logging.

Briefly, you have various applications on different servers log to a Message queue and run a Windows service that will read from the queue and populate a database. You then create a log viewer application which you can use to analysis the logs. With this set up, you can implement a number of useful processes to deal with logging.

Benefits

• No more dealing with large log files
• Template for the type of logs you collect will make your logging a lot more uniform in content and easy to read
• Clean and maintain your logs efficiently and effectively
• Search and query your logs
• Having a central logging repository enables you to report and analyze the health of your applications
• Developers need not be the only ones required to analyze a problem, other levels can access this information via the web viewer application

For more information on how to implement the above, have a look at this post.

CodeProject