External Functions Call Tracer for Executable Linked Shared Libraries

• Download convert.zip - 767 B
• Download tracer.zip - 8.4 KB

Introduction

This utility which is composed of a script and a small shared library code enables the user to apply the GCC provided functions instrumentation feature to trace all functions call in a process in the order they were called without making any modifications in the original source code which is being debugged/traced.

As a small example - given the code:

extern void goo(); //Implemented in a shared object linked with this exe...
void foo(){...}	//implemented in the exe...
	 
int main() {
    foo();
    goo();
	
    return 0;
}

The output from running the program will be a trace.out file, given the file as an input to the script, the script will print the following:

Thread 70431040 --> main at 2013-12-07T15:36:30+0000, called from __libc_start_main (??:0)
Thread 70431040 --> foo() at 2013-12-07T15:36:30+0000, called from main (main.cpp:86)
Thread 70431040 <-- foo() at 2013-12-07T15:36:30+0000
Thread 70431040 --> goo() at 2013-12-07T15:36:30+0000, called from main (main.cpp:88)
Thread 70431040 <-- goo() at 2013-12-07T15:36:30+0000
Thread 70431040 <-- main at 2013-12-07T15:36:30+0000

The utility will handle coherently functions called and defined both in the executable file and in any shared objects it's linked against.

The user can control the trace granularity by adding -finstrument-functions-exclude-file-list=file,file,... compilation flag.

The function names in the output are demangled.

P.S.: This is a modification of http://balau82.wordpress.com/2010/10/06/trace-and-profile-function-calls-with-gcc/.

Using the Code

To use the utility, the following steps are to be done:

1. Build the trace library from the source tracer.cpp file with the following command: g++ -g -shared -fpic -pthread tracer.cpp -o libtrace.so -ldl
2. Compile the traced code (executable or shared libraries) with g -rdynamic -finstrument-functions flags.
3. Link everything including the traced library, insert the tracer library link command after any of the traced libraries.

P.S.: The tracer functionality has a heavy performance impact, mainly due to the dladdr function use and secondly the frpintf (checked with valgrand's callgrind tool. I still have no clue how to improve it and will be happy to get any advice. 

Due to this issue, the tracer code can be turned on/off during runtime by setting/unsetting RUN_TRACE environment variable - this is accomplished with GDB attaching to the running traced process just before the scenario that needs to be debugged - http://unix.stackexchange.com/questions/38205/change-environment-of-a-running-process.  

P.P.S The -g flag which adds debug symbols to the tracee code inccures a big increase in resultting obj/exe/so files size. The utility will work fine without it, just  the source line numbers won't be there. 

P.P.P.S Notice that if in the final output you get a lot of same symbol repeating itself and it's clearly not
part of any library you trace, try to add this symbol as another OR clause to the script in row 44 - 

"if test "${MANGALEDCNAME}" = "(null)" -o "${MANGALEDCNAME}" = "__gxx_personality_v0""  

I suspect this might be a result of dladdr mistakenly resolving an executable's text section address to some adjacent (irrelevant) symbol. 

Another update - In case you don't see the trace.out file grow in size (stays zerro size) try to define this variable in your env - LD_PRELOAD="path to libtrace.so"