Easy to use Wrapper (DLL) for Intel's OpenCV Library with Examples

• Download library (including OpenCV DLL's) - 1.6 MB
• Download library source project - 72.48 KB
• Download sample executables - 368.79 KB
• Download sample source projects - 71.57 KB

Introduction

If the first thing you want to do is try out some examples please start with reading the Install section.

This is my first contribution for this great forum and I hope the wrapper I'm introducing will be useful for those who are friends of the C# language and are occupied in the area of image processing.

Please excuse my mistakes (especially with English grammar) but I will be grateful for references and suggestions. I anticipate that the presented library is not yet complete. With sufficient interest I will gladly extend the missing functions and descriptions.

Some years ago I decided to change from C++ (MSVS 6.0) and Java (Eclipse) to C#. I was surprised at how easy it was to pick up, even for understanding complex tasks. At the same time I was seized by the desire to port some of my older C++ image processing projects where I used parts of Intel's OpenCV Library.

Porting my software wasn't the hardest obstacle, but I wasn't familiar with interop mechanism which was need to adopt Intel's library. After searching the net, I found two possible solutions. On one side was SharperCV and on the other EmGuCv. Both projects are quite extensive and cover a rich set on functionality and I have to acknowledge that I got many hints from it and also used some small pieces of code. The underlying reason why I decided to write my own library was to learn more about Platform invoke (P/invoke) and to create a library which is usable, where one one can write method calls in a "c-style" manner. Ideally it should take as little effort as possible for a user to learn somthing about the wrapper and it should be sufficient to only read the original OpenCV documentation and of course some of my documentation.

Some Features

• The wrapper is provided in the form of a dynamic link library. (DLL)
• All exported methods (e.g. cvCanny,...), the most important macros, constructors, constant definitions and callbacks are visible through on class. (e.g.: cvlib.Functionname(...), cvlib.CV_LOAD_IMAGE_COLOR).
• All structure definitions are usable in a familiar way (e.g. IplImage img;, CvMat mat;).
• Special function parameters like pointers are passed by IntPtr, and for casting additional support is available (a second tools library, see cvtools)
when necessary.
• The conversion to and from drawing types like Bitmap is integrated in the Library and casting operators to and from Bitmap are available.
• For error handling, the provided mechanisms of the OpenCV Library can be used. There is no support for exceptions (still not).
• Callbacks are handled by predefind delegates. Use trackbars in a child window, mouse callbacks and error handlers in a familiar way.
• The user has to take care freeing the unmanaged memory. This is simply done by call of the appropriated OpenCV functions, e.g. CvReleaseImage(...).

The OpenCV Library has a very extensive amount of highly developed algorithms (more than 400 functions, hundreds of macros, myriads of constants etc.) so it seemed nearly impossible for a single person to encapsulate all of this. Additionally, there is a high probability of overloading and default values.

The idea was to write a simple HTML parser which, one one hand was able to extract all function prototypes and descriptions from the OpenCV HTML documentation, and on the other able to generate the necessary code by using predefined rules. With this, I could save a lot of time but because I hadn't the time to write a perfect converter and the resulting code was not perfect, much additional handwork was necessary.

Environment and IDE

• Visual Studio 2005 Standard Edition
• Version 8.050727.762 (SP.050727-7600)
• Microsoft .Net Framework 1.1. / 2.0 / 3.0

Content of the Downloads

• cvlib_bin.zip - DLL binaries (including OpenCV DLL's)
• cvlib_src.zip - DLL sources
• samples_bin.zip - Examples binaries
• samples_src.zip - Examples sources

Installation

• Samples
1. Download "cvlib_bin.zip", "samples_bin.zip" and extract zip's in a folder of your choice. You will get 3 directories denoted "cvlib_bin", "bigsample_bin", "smallsample_bin".
2. Copy all DLLs from "cvlib_bin" to the sample folders.
3. Run samples. If you want to try out the Camera Calibration (within the 'bigexample application') there are some images in the folder you can use for teaching and one for undistorting demonstration.
• Sources
1. Download "cvlib_bin.zip" (if not already done), "cvlib_src.zip", "samples_src.zip" and extract the zip files in a folder of your choice. You will get directories denoted "cvlib_bin", "cvlib_src", "bigsample_src" and "smallsample_src".
2. For compilation the wrapper goes to the directory "cvlib_src," and click the "openCV.csproj" file to open Visual Studio. Compile the library.
3. For the examples go to the corresponding folders ("bigsample_src", "smallsample_src") and create bin\Debug, and bin\Release directories.
4. Copy the DLL - and XML - files from "cvlib_bin" to the debug and release folder you just created. The XML — files are necessary if you want to read the methods tool tip messages generated by the VS intellisense feature.
5. Go to your studio environment for a project and look at the left side for references, click with the right mouse button and choose "Add Reference".
6. From the menu select the "Browse" tab page, go to the bin/debug folder of your project (depending if you are in the debug or release modules) and add only the cvlib.dll.
   
7. For your own projects you also need to add a using-reference ("using OpenCV;") at the top of your *.cs where you want to use the library.

Using of the Library - A Short Tutorial

The example illustrates how to implement a simply canny edge detector with its image output in a picture box control and a OpenCV image window. It will then later being extended with error handling, a track bar and it will be shown how to retrieve mouse coordinates from an external OpenCV window.

Your first step is to create a windows form application and add the DLL's using a directive as described above. Add two buttons and a picture box control to your form. The one button is simply to exit the application the other is intended to load a image file in your application. Add an "openFileDialog" Control from the Toolbox. Create the event handler for the buttons (by double clicking on the control). In the "Open File" button eventhandler add the following code:

private void buttonFile_Click(object sender, EventArgs e)
{
  IplImage img;

  if (openFileDialog1.ShowDialog() == DialogResult.OK)
  {
    // Load image from file
    img = cvlib.CvLoadImage(openFileDialog.FileName, cvlib.CV_LOAD_IMAGE_COLOR);
                
    // Create external window
    cvlib.CvNamedWindow("MyWindow", cvlib.CV_WINDOW_AUTOSIZE);
                
    // Show the image in external Window
    cvlib.CvShowImage( "MyWindow", ref img );
                
    // Show the image additional in the Picture Box Control
    this.pictureBox1.Image = (Bitmap)img;
  }
}

It is assumed that the Visual Studio Code Generator adds the name "openFileDialog1" for the file dialog. As you can see its quite simple to use the library calls. For objects which will be read or written it is necessary to use the "ref" keyword. Maybe you noticed that I didn't take care to release the unmanaged memory (and of course didn't dispose the bitmap image). You are responsible to do this any time the image has to be released (or any other unmanaged memory) by using the desired calls (see below).

In the next step we will add the canny edge detector. But to achieve this we first have to create a grey level image and apply a colour conversion.

private void buttonFile_Click(object sender, EventArgs e)
{
  IplImage img, gray;

  if (openFileDialog1.ShowDialog() == DialogResult.OK)
  {
    // Load image from file
    img = cvlib.CvLoadImage(openFileDialog.FileName, cvlib.CV_LOAD_IMAGE_COLOR);
                
    // create grey channel image
    gray = cvlib.CvCreateImage(new CvSize(img.width, img.height),
        (int)cvlib.IPL_DEPTH_8U, 1);
                
    // color conversion
    cvlib.CvCvtColor(ref img, ref gray, cvlib.CV_BGR2GRAY);
                
    // apply the operator
    cvlib.CvCanny(ref gray, ref gray, 100, 100, 3);
                
    // Create external window
    cvlib.CvNamedWindow("MyWindow", cvlib.CV_WINDOW_AUTOSIZE);
                
    // Show the edge image in external Window
    cvlib.CvShowImage( "MyWindow", ref gray );
                
    // Show the input image in the Picture Box Control 
    this.pictureBox1.Image = (Bitmap)img;
  }
}

Did it work? I hope. Finally, let's add a track bar control to the external window and receive mouse events and display the mouse coordinates in a label. Add a label control for the text output anywhere on your form window.

Because the C# language uses so called delegates for receiving events we must follow this conventions. In the presented cvlib.dll predefined delegates are already implemented, so one has only to instantiate it. We do this by adding two members and allocate it in the forms constructor.

public partial class Form1 : Form
{
  ...
  
  private cvlib.OnTrackbarChangeCallback onChange;
  private cvlib.OnMouseCallback onMouse;
  
  ...
        
  public Form1()
  {
    InitializeComponent();
    
    ...
    
    onChange = new cvlib.OnTrackbarChangeCallback(OnChange);
    onMouse = new cvlib.OnMouseCallback(OnMouse);

    ...
  }        
  ...
}

Now we can add eventhandler for the callbacks. The names for the handlers (method names) are not restricted. Feel free to select a name of your choice. In the example I used "OnChange" and "OnMouse."

More important than the names are the parameter lists for the handlers. Either you watch the line that will pop up when you typed the opening brace (in Visual Studio with intellisense on) or you look at the OpenCV documentation.

// this Method is called when Trackbar value changed 
private void OnChange(int value)
{
  // Set the labels text with trackbar value
  labelValue.Text = value.ToString();
}

// This Method is called when the mouse is moved around the child window  
private void OnMouse(int evnt, int x, int y, int flags, IntPtr param)
{
  labelValue.Text = 
  "Event: " + evnt.ToString() + 
  "\nx=" + x.ToString() + 
  "\ny=" + y.ToString() + 
  "\nFlags: " + flags.ToString();
}

Extend your File Dialog Method now with the following lines

// Position of the trackbar
int value = 0;
    
if (openFileDialog1.ShowDialog() == DialogResult.OK)    
{
  ...
        
  // Create external window
  cvlib.CvNamedWindow("MyWindow", cvlib.CV_WINDOW_AUTOSIZE);
        
  // Create the trackbar
  cvlib.CvCreateTrackbar("TrackbarName", "MyWindow", ref value, 100,
      onChange); // <-- add
        
  // Set the mouse callback eventhandler
  cvlib.CvSetMouseCallback("MyWindow", onMouse, IntPtr.Zero);    // <-- add

  ...
}

The "value" variable should be updated every time the trackbar's value has been changed. For me this doesn't work correctly. The identifier for the callback routines are passed as parameters without the <ref> keyword. TrackbarName is the leading string of the ruler and specifies its function. Because OpenCV identifies the relationships between windows by strings, the window name "MyWindow" is an additional parameter.

DLL Calls within the Wrapper and Exported Functions

Most of my internal DLL-calls follow simple rules. If the called function returns a structure pointer it is converted to the desired structure definition by using the Marshal.PointerToStructure conversion. The returned pointer is saved as an additional parameter in the structure definitions and used for releasing the allocated memory when the corresponding release call is processed (like cvlib.CvReleaseImage(...)). All primitive data types including one dimensional arrays are passed directly or by reference. Strings are converted by using the desired convention (LPStr). Structures (like the IPL image structure) passed by reference to avoid read / write access violations. Void — pointers are passed by IntPtr. To converts these arguments use the functions available in the cvtool class. The following example shows how to allocate data for the CvMat component.

// a array with doubles
double[] a = { 1.1, 2.2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12 };

// create a matrix with 3 rows and 4 columns
CvMat ma = cvlib.CvCreateMat(3, 4, cvlib.CV_64FC1);
            
// as long as the handle (h) is not disposed, 
// the garbage collector will not touch the double array 
GCHandle h;

// fill our matrix with data
cvlib.CvInitMatHeader(ref ma, 3, 4, cvlib.CV_64FC1, cvtools.Convert1DArrToPtr(a, out h),
    cvlib.CV_AUTO_STEP);

//...
// do anything with the matrix
//...

// dispose
cvtools.ReleaseHandel(h);

The cvtool Class

By using the methods of the cvtool class it is very simple to handle the c-pointer types. For most of the Methods the changes in values within the unmanaged memory will be visible in the managed environment. The follow listing shows the available conversion routines.

• Convert2DArrToPtr(byte[][] arr, out GCHandle[] handles)
• Convert2DArrToPtr(int[][] arr, out GCHandle[] handles)
• Convert2DArrToPtr(float[][] arr, out GCHandle[] handles)
• Convert2DArrToPtr(double[][] arr, out GCHandle[] handles)
• Convert1DArrToPtr(byte[] arr, out GCHandle handle)
• Convert1DArrToPtr(int[] arr, out GCHandle handle)
• Convert1DArrToPtr(float[] arr, out GCHandle handle)
• Convert1DArrToPtr(double[] arr, out GCHandle handle)
• Convert1DArrToPtr(CvPoint2D32f[] arr, out GCHandle handle)
• ConvertStructureToPtr(object structure, out GCHandle handle)
• ConvertPtrToStructure(IntPtr p, Type typeOfStructure)
• ConvertPtrToArray(IntPtr data, byte[] arr)
• ConvertPtrToArray(IntPtr data, float[] arr)
• ConvertPtrToArray(IntPtr data, double[] arr)
• ConvertPtrToArray(IntPtr data, int[] arr)
• ReleaseHandels(GCHandle[] h)
• ReleaseHandel(GCHandle h)

Overloading

One of the biggest problems is the possibility of overloading OpenCV functions due to the number of combinations of structure types and default values. So I decided to only support a few combinations and added partial overloading for default values. Maybe I will build a better support for the ubiquitous CvArr type if i will get a good idea to do this. Help here is very appreciated.

Error Handling

The OpenCV Library offers two major, different methods to get information if an error occurs. The first way is to ask for the Error status, or you get informed about errors within a callback. Furthermore, there is a way to distinguish whether you want to continue program execution or suspend the application.

It is important to know, that, when the program is internally terminated some unwanted behaviour can occur and of course this can also happen if you continue because some values are undefined. OpenCV does not support exceptions. So if you put a try-catch block and a error happens in a function call, a message box pops up by default and depending on which button you choose (cancel, try again, ignore) the application exits or crashes in most cases and the try-catch block is ignored.

The first decision you have to make is regarding the error mode. You have three options (some text copied from OpenCV-documentation):

• Leaf
  The program is terminated after error handler is called. This is the default value. It is useful for debugging, as the error is signalled immediately after it occurs. However, for production systems other two methods may be preferable as they provide more control.
  You can define your own error handler or use the predefined handler (see below)
• Parent
  The program is not terminated, but the error handler is called. The stack is unwinded (it is done w/o using C++ exception mechanism). User may check error code after calling CxCore function with cvlib.CvGetErrStatus() and react.
• Silent
  Similar to Parent mode, but no error handler is called.

If you use Silent mode you may check after calling a function with cvlib.CvGetErrStatus() and react. For defining you own error handler (this is suggested) you have to redirect the error by using the cvlib.RedirectError(...) method where you pass your own error handler as an argument. The usage is similar to a mouse or trackbar callback (with a delegate).

The following code snippet illustrates the variants (using the example from above).

public partial class Form1 : Form
{
  // define the error delegate member
  cvlib.OnErrorCallback onError;

  // Constructor
  public Form1()
  {
    InitializeComponent();
    
    // create a instance and pass your error handler methode name
    onError = new cvlib.OnErrorCallbackDelegate(OnError);
            
    // advise the CvLib to work in Parent mode (call error handler but dont exit) 
    cvlib.CvSetErrMode(cvlib.CV_ErrModeParent);
            
    // redirect the error to use your error handler against the predefind handler
    // (Message Box)
    cvlib.CvRedirectError(onError);
  }
        
  private void buttonFile_Click(object sender, EventArgs e)
  {
    ...
            
    if (openFileDialog1.ShowDialog() == DialogResult.OK)
    {
      ...
                
      // apply the operator
      cvlib.CvCanny(ref gray, ref gray, 100, 100, 3);
                
      // check for error (variant 1)
      if (cvlib.CvGetErrStatus() != 0)
      {
        // do some clean up
        cvlib.CvReleaseImage(ref img);
                    
        // !! important set error status back
        cvlib.CvSetErrStatus(0);
                    
        // return for example or throw exeption
        return;
      }
    }
            
  ...
}
        
// your error handler
private int OnError(int status, string func_name,
string err_msg, string file_name, int line)
{
  MessageBox.Show(
  "Status: " + cvlib.CvErrorStr(status) + 
  "\nIn Function: " + func_name + 
  "\nMessage: " + err_msg + 
  "\nIn File: " + file_name + 
  "\nOn Line: " + line.ToString(), "CV-Error", MessageBoxButtons.OK,
      MessageBoxIcon.Error);
                
  // a return value > 0 means application is suspended, avoid this!!
  return 0;
}

Low Level Operations

Low level operations are illustrated in the next sample. It will be shown how to access and manipulate either Image/Matrix data by using macros or by directly accessing the raw data by using unsafe code. For unsafe operations it is necessary to turn on this option in Visual Studio. You can do this by clicking the check on the "allow unsafe code" checkbox in the build tab of your project options. For accessing raw data the OpenCV Library offers some macros. This are:

• CvPtr*D
• CvSet*D
• CvGet*D
• CvGetReal*D
• CvSetReal*D

Here are some examples of how to read/write Data by using one of the macros.

// Create a matrix
CvMat m = cvlib.CvCreateMat(3, 3, cvlib.CV_64FC1);

// set data
cvlib.CvSetReal2D(ref m, 0, 0, 12.4);
cvlib.CvSetReal2D(ref m, 0, 1, 2.4);
...

// read data
double res = cvlib.CvGetReal2D(ref m, 0, 0);

The next example shows how to colourize an image area by using unsafe access. Suppose we have a colour image (IplImage image) and want to fill a rectangular area with top left at P(10, 10) and width and height = (100, 100). The image data base address is given by the imageData IntPtr class. We will get the a actual address for writing data by offsetting this base address. This is similar to: image[y * imageWidthStep + x]. imageWidthStep may be shorter than actual image width because of boxing or an ROI.

unsafe
{
  ...
        
  // get base address
  int baseAddress = image.imageData.ToInt32();
        
  // iterate
  for (int x = 10; x < 100; x++)
  {
    for (int y = 10; y < 100; y++)
    {
      byte* dst = &(((byte*)(baseAddress + image.widthStep * y))[x * 3]);
      dst[0] = 0; dst[1] = 0; dst[2] = 255;
    }
  }
        
  ...
}

Table with Conversions for Interop - p/Invoke which has Been Used

The table shows the used analogies for declaration of OpneCV API types in C#. It is regarded only to the arguments in method calls for the OpenCV library and has not been tested for common Windows API calls. What is explained is HOW the arguments are passed and gives hints when the cvtool-class can be used for adapting.

API-Type	Passed in C# as	Explantation/Hints
int, uint, uint64, float, double	int, uint, UInt64, float, double	uint64 = cvRNG is the seed for the Random Number Generator
int*	ref int, int[]	<ref> for reading/writing single value, int[] for reading/writing one dimensional arrays
float*	ref float, float[]	<ref> for reading/writing single value, float[] for reading/writing one dimensional arrays
Struct	struct	For structs that contains only primitive datatypes for reading.
Struct*	ref Struct	For structs that contains pointers and where the struct data may modified.
Struct**	ref IntPtr	Most in case that data will be deallocated.
char*	[MarshalAs(UnmanagedType.LPStr)] String	CharSet = CharSet.Ansi. In cases that the function reads a string.
char*	ref StringBuilder	CharSet = CharSet.Ansi. In cases that the function allocates a string and return it.
void*	IntPtr	Use cvtools.Convert1DArrayToPtr().
void**	ref IntPtr	Use cvtools.Convert2DArrayToPtr(). or pass structure by using the <ref> keyword

Microsoft describes the analogies in the .NET Framework documentation under the Kewords "Programing with with .NET Framework", "Communication with unmanaged code", "Interop-Marshalling", Marshalling data with Platform invoke" and "Datatypes for Platform invoke".

Available Functions in the Wrapper Library

• cxcore is most complete
• cv is most complete
• highgui is most complete
• Machine learning and cvaux is open

To Do

There is so much to-do. Any help is always welcome!

• Documentation
• Overloading
• Adding missing functions
• Test routines

The "Big-example" Application

Features

• Load and save images of different image formats supported by the OpenCV Library.
• Choose from a set of image processing operators which are directly adapted from the samples distributed with latest OpenCV package.
• Display live video from webcam or file by using HighGUI functions. Simple real time processing of your own code.
• See how to implemet a modless dynamic configurable dialog
• Implemented image processing operations are for example: Several low level filters, Camera Calibration, Watershed Trafo, Pyramids, Hough Trafo, Histogram and much more.

The presented application is intended for reference or to be a good starting place to build your own Application. Inside the mainForm.cs file all properties and methods are arranged by regions. Within Visual Studio IDE you should first collapse all definitions to get a good overview. Most importantly, definitions are:

• Process Run
• Process Menu
• Video Start / Stop / Timer

Under "Process Run" you will find the eventhandler that is executed when the "Run" button was pushed. The if-constructs check for the actual selected entry in the 'operations' combo-box and branches to the corresponding image processing method.

The 'operations' combo-box SelectedIndexChanged-Method has been placed within the "Process Menu" region. Every time the index is changed this method creates a dynamic dialogbox with corresponding controls adequate to the selected image processing task.

For every image processing operation I wrote a separeted function or class placed in different files. So it is simple to find the corresponding code. Each of the functions takes only one parameter, the actual, loaded image in IPL format. Depending on the selected mode the resulting image of an operation will be displayed in a new tab or the actual tab page.

History

• August, 2008: Library first published.