Click here to Skip to main content
Click here to Skip to main content

G3log, asynchronous logging the easy way

, 20 Aug 2014 Public Domain
Rate this:
Please Sign up or sign in to vote.
Asynchronous, "crash safe" logging with dynamic logging sinks

Introduction

G3log is an asynchronous "crash safe", logger with support for adding custom made logging sinks.   It is open source and cross platform, currently used on Windows, Linux and OSX.   G3log was built to be as safe to use as a synchronous logger but blazing fast with all the slow logging work done in background threads. 

It comes with an optional logging sink to save logs to file and supports easy adding of custom made logging sinks. 

 

G3log example with possible logging sinks vs a traditional logger
Traditional logger vs the asynchronous g3log. G3log features possible logging sinks that are currently used by some g3log users.

G2log -> G3log

G3log is open source and cross-platform. G3logs builds on the asynchronous logger g2log that was released in 2011 and previously presented here at CodeProject.

An interesting  performance benchmarking for g2log should be seen in comparison with g3log, which is roughly 30% faster than g2log.  The more LOG calling threads, the greater the performance difference in g3log’s favor

G3log features compelling functionality such as:

  • Logging and design-by-contract framework
  • LOG calls are asynchronous to avoid slowing down the LOG calling thread
  • LOG calls are thread safe
  • Queued LOG entries are flushed to log sinks before exiting so that no entries are lost at shutdown
  • Catching and logging of SIGSEGV and other fatal signals ensures controlled shutdown
  • On Linux/OSX a caught fatal signal will generate a stack dump to the log
  • G3log is cross platform, currently in use on Windows, various Linux platforms and OSX
  • Completely safe to use across libraries, even libraries that are dynamically loaded at runtime
  • Significant performance improvement compared to the snappy g2log.

 

Using  g3log

g3log uses level-specific logging. This is done without slowing down the log-calling part of the software. Thanks to the concept of active object [1][2] g3log gets asynchronous logging – the actual logging work with slow disk or network I/O access is done in one or several background threads

Example usage

Optional to use either streaming or printf-like syntax

Conditional LOG_IF as well as normal LOG calls are available. The default log levels are: DEBUG, INFO, WARNING, and FATAL.

LOG(INFO) << "streaming API is as easy as ABC or " << 123;
 
// The streaming API has a printf-like equivalent
LOGF(WARNING, "Printf-style syntax is also %s", "available");

 

Conditional Logging

LOG_IF(DEBUG, small < large) << "Conditional logging is also available. 
 
// The streaming API has a printf-like equivalent
LOGF_IF(INFO, small > large, 
             "Only expressions that are evaluated to true %s, 
             "will trigger a log call for the conditional API")

 

Design-by-Contract

CHECK(false) will trigger a “fatal” message. It will be logged, and then the application will exit. A LOG(FATAL) call is in essence the same as calling CHECK(false).

CHECK(false) << "triggers a FATAL message"
CHECKF(boolean_expression,"printf-api is also available");

 

Initialization

A typical scenario for using g3log would be as shown below. Immediately at start up, in the main function, g2::LogWorker is initialized with the default log-to-file sink.

// main.cpp
#include<g2log.hpp>
#include<g2logworker.hpp>
#include <std2_make_unique.hpp>
 
#include "CustomSink.h" // can be whatever
 
int main(int argc, char**argv) {
   using namespace g2;
   std::unique_ptr<LogWorker> logworker{ LogWorker::createWithNoSink() }; // 1
   auto sinkHandle = logworker->addSink(std2::make_unique<CustomSink>(),  // 2
                                          &CustomSink::ReceiveLogMessage); // 3
   g2::initializeLogging(logworker.get());  // 4
  1. The LogWorker is created with no sinks.
  2. A sink is added and a sink handle with access to the sinks API is returned
  3. When adding a sink the default log function must be specified
  4. At initialization the logging must be initialized once to allow for LOG calls

G3log with sinks

Sinks are receivers of LOG calls. G3log comes with a default sink (the same as G2log uses) that saves LOG calls to file. A sink can be of any class type without restrictions as long as it can either receive a LOG message as a std::string or as a g2::LogMessageMover.

The std::string option will give pre-formatted LOG output. The g2::LogMessageMover is a wrapped struct that contains the raw data, g2::LogMessage. Use this if you prefer your own custom handling and formatting in your sink.

Using g2::LogMessage is easy:

// example similar to the default FileSink. It receives the LogMessage
//        and applies the default formatting with .toString()
void FileSink::fileWrite(LogMessageMover message) {
   ...

   std::string entry =  message.get().toString(); 
   ...

}

 

Sink Creation

When adding a custom sink a log receiving function must be specified, taking as argument a std::string for the default log formatting look or taking as argument a g2::LogMessage for your own custom made log formatting.

auto sinkHandle = logworker->addSink(std2::make_unique<CustomSink>(), // 1, 3
                                     &CustomSink::ReceiveLogMessage); // 2
  1. A sink is owned by the G3log and is transferred to the logger wrapped in a std::unique_ptr
  2. The sink’s log receiving function is given as argument to LogWorker::addSink
  3. LogWorker::addSink returns a handle to the custom sink.

 

Calling the custom sink

All public functions of the custom sink are reachable through the handler.

// handle-to-sink calls are thread safe. The calls are executed asynchronously
std::future<void> received = sinkHandle->call(&CustomSink::Foo, some_param, other);

 

Code Examples

Example usage where a custom sink is added. A function is called though the sink handler to the actual sink object.

// main.cpp
#include<g2log.hpp>
#include<g2logworker.hpp>
#include <std2_make_unique.hpp>
 
#include "CustomSink.h"
 
int main(int argc, char**argv) {
   using namespace g2;
   std::unique_ptr<LogWorker> logworker{ LogWorker::createWithNoSink() };
   auto sinkHandle = logworker->addSink(std2::make_unique<CustomSink>(),
                                          &CustomSink::ReceiveLogMessage);
 
   // initialize the logger before it can receive LOG calls
   initializeLogging(logworker.get());
   LOG(WARNING) << "This log call, may or may not happend before"
                << "the sinkHandle->call below";
 
 
   // You can call in a thread safe manner public functions on your sink
   // The call is asynchronously executed on your custom sink.
   std::future<void> received = sinkHandle->call(&CustomSink::Foo, 
                                                 param1, param2);
 
   // If the LogWorker is initialized then at scope exit the g2::shutDownLogging() will be called. 
   // This is important since it protects from LOG calls from static or other entities that will go out of
   // scope at a later time. 
   //
   // It can also be called manually:
   g2::shutDownLogging();
}

 

Example usage where a the default file logger is used and a custom sink is added

// main.cpp
#include<g2log.hpp>
#include<g2logworker.hpp>
#include <std2_make_unique.hpp>
 
#include "CustomSink.h"
 
int main(int argc, char**argv) {
   using namespace g2;
   auto defaultHandler = LogWorker::createWithDefaultLogger(argv[0], 
                                                 path_to_log_file);
 
   // logger is initialized
   g2::initializeLogging(defaultHandler.worker.get());
 
   LOG(DEBUG) << "Make log call, then add another sink";
 
   defaultHandler.worker->addSink(std2::make_unique<CustomSink>(),
                                  &CustomSink::ReceiveLogMessage);
 
   ...
}

 

It is easy to start using the logger anywhere in your code base.

// some_file.cpp
#include <g2log.hpp>
void SomeFunction() {
   ...
   LOG(INFO) << "Hello World";
}

 

g3log API

In addition to the logging API: LOG, LOG_IF, CHECK (and the similar printf-calls) there are a few functions that are helpful for using and tweaking g3log.

Initialization

// g2log.hpp
initializeLogging(...) 

Dynamic log levels

I.e. disabling/enabling of log-levels at runtime. Enabling of this feature is done by a
#define G2_DYNAMIC_LOGGING.

// g2loglevels.hpp
void setLogLevel(LEVELS level, bool enabled_status);
bool logLevel(LEVELS level); 

 

Sink Handling

See g2logworker.hpp for adding a custom sink, or accessing the default filesink.

std::unique_ptr<SinkHandle<T>> addSink(...)
static g2::DefaultFileLogger createWithDefaultLogger(...)
static std::unique_ptr<LogWorker> createWithNoSink();

 

Internal Functions

See g2log.hpp for several internal functions that usually don’t have to be used by the coder but can if g3log has to be tweaked for special shutdown considerations. 

// will be called when the LogWorker goes out of scope
shutDownLogging() 
 
// for unit testing, or customized fatal call handling
void changeFatalInitHandlerForUnitTesting(...) 

 

Where to get it

Please see https://bitbucket.org/KjellKod/g3log

Thanks

Thanks to great community feedback from users of g2log. With your feedback and eager beta testing g2log could be further improved into the current release of g3log.   

 

Enjoy

Kjell (a.k.a. KjellKod)

License

This article, along with any associated source code and files, is licensed under A Public Domain dedication

Share

About the Author

KjellKod.cc
Software Developer (Senior) LogRhythm
United States United States
Enjoying Colorado! Family and intense software development.
 
Kjell is a driven developer with a passion for software development. His focus is on coding, boosting software development, improving development efficiency and turning bad projects into good projects.
 
Kjell was twice national champion in WUKO and semi-contact Karate and now he focuses his kime towards software engineering, everyday challenges and moose hunting while not being (almost constantly) amazed by his daughter and sons.

Comments and Discussions

 
QuestionCan I use g3log in a MFC program? Pinmemberkonkoner30-Oct-14 18:03 
AnswerRe: Can I use g3log in a MFC program? PinmemberKjellKod.cc30-Oct-14 20:44 
GeneralRe: Can I use g3log in a MFC program? Pinmemberkonkoner30-Oct-14 23:19 
GeneralRe: Can I use g3log in a MFC program? PinmemberKjellKod.cc31-Oct-14 3:15 
GeneralRe: Can I use g3log in a MFC program? Pinmemberkonkoner9-Nov-14 22:39 
QuestionGreat work! PinmemberDilukhin14-Sep-14 20:46 
AnswerRe: Great work! PinmemberKjellKod.cc15-Sep-14 6:07 
QuestionIs it possible to set different log file? Pinmemberplasticbox27-Aug-14 20:19 
AnswerRe: Is it possible to set different log file? [modified] PinmemberKjellKod.cc28-Aug-14 0:28 
QuestionVery interesting Pinmemberyarp24-Aug-14 21:18 
AnswerRe: Very interesting PinmemberKjellKod.cc2-Sep-14 19:02 
GeneralWell done Kjell! PinmentorEspen Harlinn21-Aug-14 10:21 
GeneralRe: Well done Kjell! PinmemberKjellKod.cc21-Aug-14 14:54 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

| Advertise | Privacy | Terms of Use | Mobile
Web01 | 2.8.141223.1 | Last Updated 21 Aug 2014
Article Copyright 2014 by KjellKod.cc
Everything else Copyright © CodeProject, 1999-2014
Layout: fixed | fluid