You can use Assembly.Load
] and then reference the classes via reflection - but it's a fair amount of work and you can't use class names directly in your code in the usual way.
If you want to say things like:
MyAssembly.MyClass mc = new MyAssembly.MyClass();
Then you have to add a reference.