A Microsoft tool named RegMaid does the same thing as what you want to do. The nice thing is that it comes with the full C++ source code. As you can see there is lot involved in removing ActiveX controls (CLSID, AppID, Interfaces, Typelibs etc). Download the tool and the source code here:
The code may require some tweaking to be build in latest versions of Visual Studio, and of course if you're trying to use VB.NET you'll want to keep in mind that .NET provides some additional libraries for manipulating the registry. But still, you can get a clean idea on how to write your own tool by looking at the source.