Bag-of-Features Descriptor on SURF and ORB Features (BoF-SURF and BoF-ORB)

• Download source(OpenCV_BoFORB) - 2.3 MB
• Download source (OpenCV_BoFSURF) - 1.4 MB

Introduction

Bag-Of-Feature (BoF) is a kind of visual feature descriptor which can be used in content based image or video retrieval applications. In order to obtain a BoF descriptor, we need to extract a feature from the image. This feature can be anything such as SIFT (Scale Invariant Feature Transform), SURF (Speeded Up Robust Features), and ORB (Oriented BRIEF), etc.

With this tip, the implementations of BoF-SURF and BoF-ORB are provided. This article is prepared on the request of the readers of article, "Bag-of-Features Descriptor on SIFT Features with OpenCV (BoF-SIFT)" which describes about the Bag-of-Feature descriptor with SIFT in detail. This article only includes the changes which have been made to the original code in order to adopt it to use as BoF-SURF and BoF-ORB. Please refer to the original article for more details.

Background

SURF stands for Speeded Up Robust Feature which is more like SIFT but fast in computation. SURF algorithm gets the maximum benefits from the integral image representation which helps to the speed gain in SURF. You can read more about SURF from this article in Wikipedia.

ORB stand for Oriented BRIEF is an efficient alternative to SIFT and SURF. You can read more about ORB from this research paper "ORB: an efficient alternative to SIFT or SURF".

Using the Code

With OpenCV, we can implement BoF-SURF and BoF-ORB with just a few lines of code. Make sure that you have installed OpenCV 2.3 or higher version and Visual Studio 2008 or higher. The OpenCV version requirement is a must but still you may use other C++ flavors without any problems.

The code has two separate regions that are compiled and run independently. The first region is for obtaining the set of bags of features and the other region for obtaining the BoF descriptor for a given image/video frame. You need to run the first region of the code only once. After creating the vocabulary, you can use it with the second region of code anytime. Modifying the code line below can switch between the two regions of code.

DICTIONARY_BUILD 1 // set DICTIONARY_BUILD to 1 for Step 1. 0 for step 2   

SURF Feature Extraction

In dictionary build section, we need to extract SURF feature using the following code line:

int minHessian = 400; //Hessian Threshold
//Some code lines
SurfDescriptorExtractor detector(minHessian,4,2,false); 

The SurfDescriptorExtractor constructor requires minimum Hessian threshold, number of octaves, number of octave layers, whether the extended descriptor (whether the dimensionality of the descriptor is 128 or 64) and whether the SURF is upright mode (USURF).

In the descriptor extraction section the same SURF extractor and a SURF detector as in the following code:

//create SURF feature point extracter
Ptr<FeatureDetector> detector(new SurfFeatureDetector(minHessian,4,2,false));
//create SURF descriptor extractor
Ptr<DescriptorExtractor> extractor(new SurfDescriptorExtractor(minHessian,4,2,false));  

Other codes are similar to the BoF-SIFT in the original article.

ORB Feature Extraction

For the extraction of ORB features, OrbDescriptorExtractor has been used. We use the default parameter values to create the detector object as in the following code:

OrbDescriptorExtractor detector; 

Unlike in other features, while using ORB for obtaining the BoF vocabulary file, openCV complains about assertion fail within the openCV core library due to a type mismatching problem. The workaround is as follows:

//convert featuresUnclustered to type CV_32F
Mat featuresUnclusteredF(featuresUnclustered.rows,featuresUnclustered.cols,CV_32F);
featuresUnclustered.convertTo(featuresUnclusteredF,CV_32F);
//cluster the feature vectors
Mat dictionary=bowTrainer.cluster(featuresUnclusteredF); 

In order to obtain the BoF descriptor, we need a descriptor matcher. In previous cases, the nearest neighbor technique is used. Here we use BruteForce-Hamming descriptor matcher. ORB is a binary descriptor so hamming distance for matching, works better in terms of both efficiency and performance. I experienced the type problem in this step also. Hence, I used the following code:

//convert to 8bit unsigned format
Mat dictionary(dictionaryF.rows,dictionaryF.cols,CV_8U);
dictionaryF.convertTo(dictionary,CV_8U); 
//create a matcher with BruteForce-Hamming distance
Ptr<DescriptorMatcher> matcher = DescriptorMatcher::create("BruteForce-Hamming");	 

The detector and extractor of ORB are created as in the following code:

//create SURF feature point extracter
Ptr<FeatureDetector> detector(new OrbFeatureDetector());
//create SURF descriptor extractor
Ptr<DescriptorExtractor> extractor(new OrbDescriptorExtractor());	 

Points of Interest

Please be careful when you use this code with your research, because I have used type conversion for Mat objects only for avoiding the assertion failings that occurred in the core library of openCV. Better if you can recheck the exact problem or the correct expectation of the openCV code.