Click here to Skip to main content
15,867,308 members
Articles / Programming Languages / C

Support Vector Machine Classifier

Rate me:
Please Sign up or sign in to vote.
3.82/5 (20 votes)
13 Apr 2008GPL32 min read 138.6K   8K   57   17
SVM classification class with SSE optimization support

Introduction

Support vector machine (SVM) is a non-linear classifier which is often reported as producing superior classification results compared to other methods. The idea behind the method is to non-linearly map the input data to some high dimensional space, where the data can be linearly separated, thus providing great classification (or regression) performance. One of the bottlenecks of the SVM is the large number of support vectors used from the training set to perform classification (regression) tasks. In my code, I use SSE optimization to increase performance.

Background (optional)

www.kernel-machines.org is a great source for SVM information.

Using the Code

In SVM class, I use my 2D SSE optimized vector code for faster computation. The SVMachine class contains the following functions you need to use:

  • SVMachine::SVMachine(const wchar_t* fname); ctor
  • int SVMachine::status() const; status after ctor (0 upon success and negative in case of errors)
  • unsigned int SVMachine::dimension() const; the dimensionality of the SVM
  • int SVMachine::classify(const float* x, double& y) const; to classify unknown vector x

ctor reads SVM configuration from file having the following text format:

input vector dimensionality
number of support vectors
kernel type [kernel parameter]

bias

weight
1st support vector    

weight
2nd support vector

...

For example, polynomial kernel SVM for iris data set to classify setosa from virgi consisted from 4 support vectors is presented below:

4
4
polynomial 3

1.1854890124462447

7.792329562624775e-012
51
33
17
5
 
9.9563612529691003e-012
48
34
19
2
 
1.0631195782759572e-011
51
38
19
4
 
-2.8372847134273557e-011
49
25
45
17

The SVM decision function is presented by the following formula:

SVM decision function

Where x is the input vector, alpha and y are the weights of the support vectors, having y as positive or negative class mark (+1 or -1) and b is the bias. From the iris SVM file, we can see that there are 4 four dimensional support vectors (3 first from positive class being setosa samples and the last one from negative class pertaining to virgi), the kernel is the polynomial one with 3 as the parameter, the bias is equal to 1.1854890124462447.

In my class, I use 3 kernels.

  1. linear:

    linear kernel

  2. rbf:

    RBF kernel

  3. polynomial:

    polynomial kernel

    Where param is the [kernel parameter] in the SVM file.

Typically a grid search is used to select best classification (regression) results by varying alpha and param over some range. In my iris SVM, the alphas are equal to 1 and param is the degree of polynomial.

To classify unknown vector sample x as belonging to positive or negative class, use SVMachine::classify() function. It returns +1 or -1 as the result of classification and provides to y the result of sum from the SVM decision formula.

Points of Interest

Add other kernels.

History

  • 14th April, 2008: Initial post

License

This article, along with any associated source code and files, is licensed under The GNU General Public License (GPLv3)


Written By
Engineer
Russian Federation Russian Federation
Highly skilled Engineer with 14 years of experience in academia, R&D and commercial product development supporting full software life-cycle from idea to implementation and further support. During my academic career I was able to succeed in MIT Computers in Cardiology 2006 international challenge, as a R&D and SW engineer gain CodeProject MVP, find algorithmic solutions to quickly resolve tough customer problems to pass product requirements in tight deadlines. My key areas of expertise involve Object-Oriented
Analysis and Design OOAD, OOP, machine learning, natural language processing, face recognition, computer vision and image processing, wavelet analysis, digital signal processing in cardiology.

Comments and Discussions

 
Questionplz send matlab code for svm classifier to detect skin cancer Pin
Member 1305969016-Mar-17 0:35
Member 1305969016-Mar-17 0:35 
AnswerRe: plz send matlab code for svm classifier to detect skin cancer Pin
CHill6016-Mar-17 0:39
mveCHill6016-Mar-17 0:39 
Questionprogram for svm in matlab Pin
Member 907470923-Feb-16 23:29
Member 907470923-Feb-16 23:29 
QuestionSVM Classifier Pin
மதன் குமார்6-Mar-14 5:45
மதன் குமார்6-Mar-14 5:45 
QuestionHelp with SVM syntax in C++ Pin
Noe Prins7-Dec-12 2:11
Noe Prins7-Dec-12 2:11 
Questionweight and bias Pin
lordertan113-Mar-12 11:30
lordertan113-Mar-12 11:30 
QuestionHelp me!! Pin
hassan914-Jan-12 0:55
hassan914-Jan-12 0:55 
Questionf*** Pin
nicejava26-Oct-11 21:19
nicejava26-Oct-11 21:19 
Generalcan you help me to write a program of pso trainning avm by matlab Pin
chenok12-Apr-11 22:42
chenok12-Apr-11 22:42 
Generalsource code Pin
MEDJADJ28-Mar-11 7:29
MEDJADJ28-Mar-11 7:29 
I'm a student; i want source code for SVM classification (binary SVM) to seperate two classes for image a code in c++ builder version 6.5. Please
GeneralRe: source code Pin
Member 109456531-Aug-14 6:02
Member 109456531-Aug-14 6:02 
GeneralMy vote of 2 Pin
César de Souza9-Sep-10 15:58
professionalCésar de Souza9-Sep-10 15:58 
GeneralRe: My vote of 2 Pin
Noe Prins7-Dec-12 3:42
Noe Prins7-Dec-12 3:42 
Generalonevsrest svm Pin
erolco11-Dec-09 13:41
erolco11-Dec-09 13:41 
Generalonly classifier Pin
LalitM200917-May-09 18:24
LalitM200917-May-09 18:24 
Questionhow to train the SVM Pin
huashanyjj1-Apr-09 22:46
huashanyjj1-Apr-09 22:46 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.