You could use "precompiler" directives:
#if DEBUG
using your.library.assembly
#endif
You could also late bind with your library based on phisical availablity of assembly file.
You could also use Dependancy Injection to provide "stub" types for your assembly.
I think there are several approaches you could use based on your requirements.