Backpropagation Artificial Neural Network in C++

• Download demo - 95.7 KB
• Download source - 19.5 KB

Introduction

I'd like to present a console based implementation of the backpropogation neural network C++ library I developed and used during my research in medical data classification and the CV library for face detection: Face Detection C++ library with Skin and Motion analysis. There are some good articles already present at The CodeProject, and you may consult them for the theory. In my code, I present the necessary features as input data preprocessing in the input layer with Minmax, Zscore, Sigmoidal, and Energy normalization. These parameters are obtained from the training set, and then used for preprocessing every incoming vector for classification. The console supports training data random separation to train, validation, and test sets before backpropagation training. Random separation allows to obtain a representative train set comparing performance on validation and test parts. A validation set is useful for preventing over-fitting by estimating the performance on that set. At the end of the backpropagation session, I save both network configurations, the one with the best performance on the validation set and the last training epoch configuration. For performance estimation, I use sensitivity, specificity, positive predictivity, negative predictivity, and accuracy metrics. For validation performance estimation in the case of biased data distribution (for example, in my Face Detection article, you may find that there are a lot more non-faces than faces, 19:1 ratio) I provide a geometric mean and F-measure metrics to support the scenario. To support a large amount of data vectors, I provide File Mapping based data load. That allows to map your hundreds of megs of training data to memory in 1 msec and start your training session immediately. For relatively small amounts of data, you may use a text file format. Finally, the console implementation is easier to use, you avoid a lot of mouse clicking in GUI applications, and may automate the process with batch files for choosing the right network topology, the best performance on the validation and test sets, and so on.

Background

Have a look at the CodeProject neural network articles. I also used tutorials from the generation5 site. For biased data distribution problems, I used: Evaluation of classifiers for an uneven class distribution problem. Have a look there to understand geometric mean and F-measure metrics.

Using the code

The help line to console is shown below:

 argv[1] t-train
 argv[2] network conf file
 argv[3] cls1 files [0.9]
 argv[4] cls2 files [0.1]
 argv[5] epochs num
  argv[6] [validation class]
  argv[7] [test class]
  argv[8] [validation TH 0.5]
  argv[9] [vld metric mse]
 argv[10] [norm]: [0-no], 1-minmax, 2-zscore, 3-softmax, 4-energy
 argv[11] [error tolerance cls] +- 0.05 default

 argv[1] r-run
 argv[2] network conf file
 argv[3] cls files
 argv[4] [validation TH 0.5]
 argv[5] [norm]: [0-no], 1-minmax, 2-zscore, 3-softmax, 4-energy

  ann1dn.exe t net.nn cls1 cls2 3000 [tst.txt][val.txt]
     [TH [0.5]][val type [mse]] [norm [0]] [err [0.05]] 
  ann1dn.exe r net.nn testcls [TH [0.5]] [norm [0]]

 metrics: [0 - mse]
           1 - AC
           2 - sqrt(SE*SP)
           3 - sqrt(SE*PP)
           4 - sqrt(SE*SP*AC)
           5 - sqrt(SE*SP*PP*NP*AC)
           6 - F-measure b=1
           7 - F-measure b=1.5
           8 - F-measure b=3

The minimal number of parameters to start a training session:

>ann1dn.exe t network.nn data1_file data2_file 1000

It will use the network.nn file as a neural network, and load data form data1_file and data2_file, which represents data vectors from positive and negative classes, and train it for 1000 epochs.

The neural network file format is described in my Face Detection article. To start with random initialized weights before the training session, you need to provide only the number of layers and the number of neurons per layer in that file. For example, in the demo zip, you will find a iris.nn file:

3
4 8 1

Three layers, and 4, 8, and 1 neurons per layer.

It supports two data file formats. The text one is:

vector_name class_mark
x1 x2 x3 ... xN
...

vector_name is the name of your particular vector; it should not start with a numeric element, only with a letter. class_mark is the non zero number corresponding to the class attribute: 1, 2, etc. In the console application, I use only 1 as positive class with a 0.9 desired output, and 2 as negative with 0.1 as the desired output. The next line contains the vector entries in integer or floating point format.

In the demo zip, the IRIS data is organized using a text format with four-dimensional entries:

virgi1  2
64 28 56 22 
virgi2  2
67 31 56 24 
virgi3  2
63 28 51 15 
virgi4  2
69 31 51 23 
virgi5  2
65 30 52 20 
virgi6  2
65 30 55 18 
...


setosa1  1
50 33 14 2 
setosa2  1
46 34 14 3 
setosa3  1
46 36 10 2 
setosa4  1
51 33 17 5 
setosa5  1
55 35 13 2 
...

The binary floating point file format is expedient when you have a large amount of data. The data is saved in a separate file as a sequence of floating point numbers in binary format, using 4 bytes per floating point number:

file1.dat (2x3 matrix)

[x11] [x12] [x13] [x21] [x22] [x23]

And, the dimensions of the data matrix are saved in the file with the same name but with a .hea extension.

file1.hea

2 3

In the previous example, the file file1.dat contains two three-dimensional vectors.

In that case, your data1_file or data2_file may contain the entries with the full path to the files and the class marks:

fullpath\file1.dat 1
fullpath\file2.dat 1
...

The next parameters to the console application for backprop training are optional. You may use them for validation and testing of your network, for input data normalization, and error limits during training process.

>ann1dn t network.nn data1_file data2_file 1000  vld_file tst_file 0.5 2  2

This command line demonstrates that you use your validation and test sets in the vld_file and tst_file files in text or binary format, as described above, with a validation threshold 0.5 (that is, the network output greater than 0.5 attributes a data vector to a positive class), with geometric mean of sensitivity and specificity as the performance metric for validation stopping and Zscore normalization. The allowed validation metrics are specified at the end of the console help line. If your vld_file or tst_file are empty files, then the corresponding data set will be composed of randomly selected entries from your training set, with 25% records from each class.

The last eleventh argument to the console is the error tolerance. If the difference between the desired output and the mean network output for positive and negative classes is less than the error for 10 consecutive epochs, the training stops. During the backpropagation training, if the difference between the desired output and the network output for a particular vector is less than the error you specify, the network weights are not adjusted. This allows to correct the network weight connections for the data vectors which are not yet 'memorized'.

The next example demonstrates the sample training session for the IRIS data:

>ann1dn t iris.nn setosa_versi.dat virgi.dat 200  void void 0.5 2  3

loading data...
 cls1: 100  cls2: 50  files loaded.  size: 4 samples
 validaton size: 25 12
 validaton size: 26 13
normalizing minmax...
training...
  epoch: 1  out: 0.555723 0.478843  max acur: 0.92 (epoch 1)  se:84.00 sp:100.00 ac:89.19
  epoch: 2  out: 0.582674 0.400396  max acur: 0.92 (epoch 1)  se:84.00 sp:100.00 ac:89.19
  epoch: 3  out: 0.626480 0.359573  max acur: 0.92 (epoch 3)  se:84.00 sp:100.00 ac:89.19
  epoch: 4  out: 0.655483 0.326918  max acur: 0.94 (epoch 4)  se:96.00 sp:91.67 ac:94.59
  epoch: 5  out: 0.699125 0.323879  max acur: 0.94 (epoch 5)  se:88.00 sp:100.00 ac:91.89
  epoch: 6  out: 0.715539 0.299085  max acur: 0.94 (epoch 6)  se:88.00 sp:100.00 ac:91.89
  epoch: 7  out: 0.733927 0.292526  max acur: 0.96 (epoch 7)  se:92.00 sp:100.00 ac:94.59
  epoch: 8  out: 0.750638 0.278721  max acur: 0.98 (epoch 8)  se:96.00 sp:100.00 ac:97.30
  epoch: 9  out: 0.774599 0.277550  max acur: 0.98 (epoch 8)  se:96.00 sp:100.00 ac:97.30
  epoch: 10  out: 0.774196 0.256110  max acur: 0.98 (epoch 8)  se:96.00 sp:100.00 ac:97.30
  epoch: 11  out: 0.793877 0.260753  max acur: 0.98 (epoch 8)  se:96.00 sp:100.00 ac:97.30
  epoch: 12  out: 0.806802 0.245758  max acur: 0.98 (epoch 8)  se:96.00 sp:100.00 ac:97.30
  epoch: 13  out: 0.804381 0.228810  max acur: 0.98 (epoch 13)  se:96.00 sp:100.00 ac:97.30
  epoch: 14  out: 0.814079 0.218740  max acur: 0.98 (epoch 13)  se:96.00 sp:100.00 ac:97.30
  epoch: 15  out: 0.827635 0.223827  max acur: 0.98 (epoch 13)  se:96.00 sp:100.00 ac:97.30
  epoch: 16  out: 0.832102 0.210360  max acur: 0.98 (epoch 13)  se:96.00 sp:100.00 ac:97.30
  epoch: 17  out: 0.840352 0.213165  max acur: 0.98 (epoch 17)  se:96.00 sp:100.00 ac:97.30
  epoch: 18  out: 0.848957 0.201766  max acur: 0.98 (epoch 18)  se:96.00 sp:100.00 ac:97.30
  epoch: 19  out: 0.844319 0.188338  max acur: 0.98 (epoch 19)  se:96.00 sp:100.00 ac:97.30
  epoch: 20  out: 0.856258 0.184954  max acur: 0.98 (epoch 19)  se:96.00 sp:100.00 ac:97.30
  epoch: 21  out: 0.853244 0.178349  max acur: 0.98 (epoch 19)  se:96.00 sp:100.00 ac:97.30
  epoch: 22  out: 0.867145 0.185852  max acur: 0.98 (epoch 22)  se:96.00 sp:100.00 ac:97.30
  epoch: 23  out: 0.863079 0.171684  max acur: 0.98 (epoch 23)  se:96.00 sp:100.00 ac:97.30
  epoch: 24  out: 0.870108 0.170253  max acur: 0.98 (epoch 24)  se:96.00 sp:100.00 ac:97.30
  epoch: 25  out: 0.873538 0.164185  max acur: 0.98 (epoch 25)  se:96.00 sp:100.00 ac:97.30
  epoch: 26  out: 0.871584 0.150496  max acur: 1.00 (epoch 26)  se:100.00 sp:100.00 ac:100.00
  epoch: 27  out: 0.879310 0.161155  max acur: 1.00 (epoch 26)  se:100.00 sp:100.00 ac:100.00
  epoch: 28  out: 0.879986 0.154784  max acur: 1.00 (epoch 26)  se:100.00 sp:100.00 ac:100.00
  epoch: 29  out: 0.880308 0.139083  max acur: 1.00 (epoch 26)  se:100.00 sp:100.00 ac:100.00
  epoch: 30  out: 0.890360 0.149518  max acur: 1.00 (epoch 26)  se:100.00 sp:100.00 ac:100.00
  epoch: 31  out: 0.888561 0.145144  max acur: 1.00 (epoch 26)  se:100.00 sp:100.00 ac:100.00
  epoch: 32  out: 0.880072 0.129197  max acur: 1.00 (epoch 32)  se:100.00 sp:100.00 ac:100.00
  epoch: 33  out: 0.896553 0.139937  max acur: 1.00 (epoch 32)  se:100.00 sp:100.00 ac:100.00
  epoch: 34  out: 0.893467 0.137607  max acur: 1.00 (epoch 32)  se:100.00 sp:100.00 ac:100.00
  epoch: 35  out: 0.893400 0.125793  max acur: 1.00 (epoch 32)  se:100.00 sp:100.00 ac:100.00
  epoch: 36  out: 0.905036 0.139306  max acur: 1.00 (epoch 32)  se:100.00 sp:100.00 ac:100.00
  epoch: 37  out: 0.900872 0.118167  max acur: 1.00 (epoch 32)  se:100.00 sp:100.00 ac:100.00
  epoch: 38  out: 0.909384 0.134014  max acur: 1.00 (epoch 32)  se:100.00 sp:100.00 ac:100.00
training done.
training time: 00:00:00:031

classification results: maxacur.nn
 
 train set: 49 25
   sensitivity: 100.00
   specificity: 100.00
   +predictive: 100.00
   -predictive: 100.00
      accuracy: 100.00
 
 validation set: 25 12
   sensitivity: 100.00
   specificity: 100.00
   +predictive: 100.00
   -predictive: 100.00
      accuracy: 100.00
 
 test set: 26 13
   sensitivity: 88.46
   specificity: 92.31
   +predictive: 95.83
   -predictive: 80.00
      accuracy: 89.74

classification results: iris.nn
 
 train set: 49 25
   sensitivity: 97.96
   specificity: 100.00
   +predictive: 100.00
   -predictive: 96.15
      accuracy: 98.65
 
 validation set: 25 12
   sensitivity: 96.00
   specificity: 100.00
   +predictive: 100.00
   -predictive: 92.31
      accuracy: 97.30
 
 test set: 26 13
   sensitivity: 88.46
   specificity: 100.00
   +predictive: 100.00
   -predictive: 81.25
      accuracy: 92.31

The network configuration is saved to maxacur.nn corresponding to the best performance on the validation set, and the last epoch configuration is saved to iris.nn. At the end, you may compare the results for them.

To use your trained networks for testing, just run the console with these parameters:

>ann1dn r iris.nn test_data

The test_data file is in text or binary format. You may provide as the fourth argument the threshold (default is 0.5) to obtain the ROC curve on your test set, for example, varying it between 0.0 and 1.0.

Neural Network Classes

The neural network is composed from the following classes:

• ANNetwork
• ANNLayer
• ANeuron
• ANLink

The ANNetwork class contains the implementation of the neural network for users of the library. To avoid protected interface programming for the rest of the classes, I used friends. I'll describe the library structure first, and then provide the functions you need to use from the ANNetwork class to maintain your own implementations.

The ANNetwork contains an array of ANNLayer layers. Every layer contains an array of ANeuron neuron objects, and every neuron contains arrays of ANLink input and output connections. With that design, you may arrange any desired network structure; however, in my implementation, I provide only feed-forward full connectionist structure.

The basic unit of the neural network is the neuron class, ANeuron. You may add bias or input connection to it, represented as the ANLink object.

• void ANeuron::add_bias()
• void ANeuron::add_input(ANeuron *poutn)

The bias connection always take 1.0f as an input value, as you know. With add_input(), you add connection to the neuron, supplying with its argument the neuron from the previous layer to which it connects.

void ANeuron::add_input(ANeuron *poutn)
{
    //poutn - Neuron from previous layer
    ANLink *plnk = new ANLink(this, poutn); 
    inputs.push_back(plnk);
    if (poutn)
        poutn->outputs.push_back(plnk);
}

So, every neuron 'knows' which neurons from the next layer connect to its output. The ANLink is like the 'arrow', pointing from the neuron in the previous layer, ANLink::poutput_neuron to the neuron in the next layer, ANLink::pinput_neuron.

I organize a full connectionist neural network structure in this way:

void ANNetwork::init_links(const float *avec, const float *mvec, int ifunc, int hfunc)
{
    ANNLayer *plr;      //current layer
    ANNLayer *pprevlr;  //previous layer
    ANeuron *pnrn;      //neuron pointer

    int l = 0;


    /////////////////////////input layer/////////////////////////////
    plr = layers[l++];
    swprintf(plr->layer_name, L"input layer");

    for (int n = 0; n < plr->get_neurons_number(); n++) {
        pnrn = plr->neurons[n];
        pnrn->function = ifunc;
        pnrn->add_input();
        //one input link for every "input layer" neuron

        if (avec)
            pnrn->inputs[0]->iadd = avec[n];  
        if (mvec)
            pnrn->inputs[0]->w = mvec[n];     
        else
            pnrn->inputs[0]->w = 1.0f;   
    }
    /////////////////////////////////////////////////////////////////


    ////////////////////////hidden layer's///////////////////////////////////////   1bias
    for (int i = 0; i < m_layers_number - 2 ; i++) {   //1input  [l-2 hidden]  1output
        pprevlr = plr;
        plr = layers[l++];
        swprintf(plr->layer_name, L"hidden layer %d", i + 1);

        for (int n = 0; n < plr->get_neurons_number(); n++) {
            pnrn = plr->neurons[n];
            pnrn->function = hfunc;
            pnrn->add_bias();

            for (int m = 0; m < pprevlr->get_neurons_number(); m++)
                pnrn->add_input(pprevlr->neurons[m]);
        }
    }
    //////////////////////////////////////////////////////////////////////////////


    ////////////////////////output layer/////////////////////////////////////////   1bias
    pprevlr = plr;
    plr = layers[l++];
    swprintf(plr->layer_name, L"output layer");

    for (int n = 0; n < plr->get_neurons_number(); n++) {
        pnrn = plr->neurons[n];
        pnrn->function = hfunc;
        pnrn->add_bias();

        for (int m = 0; m < pprevlr->get_neurons_number(); m++)
            pnrn->add_input(pprevlr->neurons[m]);
    }
    //////////////////////////////////////////////////////////////////////////////

}

The ANeuron functions that induce the neuron to 'fire', that is to take data from its inputs and process them to its outputs, are:

• void ANeuron::input_fire()
• void ANeuron::fire()

The first one is used for input layer neurons only. The ANLink contains an additional term, iadd, used in normalization. The last one is used for hidden and output layer neurons:

inline void ANeuron::input_fire()
{
    //input layer normalization
    oval = (inputs[0]->ival + inputs[0]->iadd) * inputs[0]->w;

    //single input for input layer neuron
    switch (function) {
    default:
    case LINEAR:
        break;

    case SIGMOID:
        oval = 1.0f / (1.0f + exp(float((-1.0f) * oval)));
        break;
    }

    //transfer my output to links connected to my output
    for (int i = 0; i < get_output_links_number(); i++)
        outputs[i]->ival = oval;
}

inline void ANeuron::fire()
{
    //oval = SUM (in[]*w[])
    oval = 0.0f;

    //compute output for Neuron
    for (int i = 0; i < get_input_links_number(); i++)
        oval += inputs[i]->ival * inputs[i]->w;

    switch (function) {
    default:
    case LINEAR:
        break;

    case SIGMOID:
        oval = 1.0f / (1.0f + exp(float((-1.0f) * oval)));
        break;
    }

    //transfer my output to links connected to my output
    for (int i = 0; i < get_output_links_number(); i++)
        outputs[i]->ival = oval;
}

Now, you have some idea of the library internals, and further, I'll describe the ANNetwork class which you can use for your own implementations.

You may load the neural network from the file, or arrange its structure specifying the number of layers and neurons per layer:

• ANNetwork::ANNetwork(const wchar_t *fname);
• ANNetwork::ANNetwork(int layers_number, int *neurons_per_layer);

int nerons_per_layer[4] = {128, 64, 32, 10};
ANNetwork *ann = new ANNetwork(4, neurons_per_layer);
ann->init_links();   //feed-forward full connectionist structure
ann->randomize_weights();

If you want your custom neural network configuration with recurrent or any other connections, you have to provide your own functions.

The ANNetwork::status() function returns the class status after construction. Negative values indicate error, 0 - network loaded from file with success, and 1 - random weights.

To train, classify, and save your network, the following functions are provided:

• bool ANNetwork::train(const float *ivec, float *ovec, const float *dsrdvec, float error = 0.05);
• void ANNetwork::classify(const float *ivec, float *ovec);
• bool ANNetwork::save(const wchar_t *fname) const;

ivec and ivec represent the input vector fed to a neural network and the output vector where it stores its results. Their dimensions should match the number of input and output neurons in the network structure. dsrdvec is the desired output vector to which it adjusts its connections to match it within the error tolerance. The ANNetwork::train() function will return true in case the backpropogation took place, or false if the network output was within error to the desired vector.

The backpropagation function uses this code:

bool ANNetwork::train(const float *ivec, float *ovec, const float *dsrdvec, float error)
// 0.0  -  1.0 learning
{
    float dst = 0.0f;

    //run network, computation of inputs to output
    classify(ivec, ovec);
    for (int n = 0; n < layers[m_layers_number-1]->get_neurons_number(); n++) {
        dst = fabs(ovec[n] - dsrdvec[n]);
        if (dst > error) break;
    }

    if (dst > error) {
        backprop_run(dsrdvec);    //it was trained
        return true;
    } else                  //it wasnt trained
        return false;
}

void ANNetwork::backprop_run(const float *dsrdvec)
{
    float nrule = m_nrule;  //learning rule
    float alpha = m_alpha;  //momentum
    float delta, dw, oval;

    //get deltas for "output layer"
    for (int n = 0; n < layers[m_layers_number-1]->get_neurons_number(); n++) {
        oval = layers[m_layers_number-1]->neurons[n]->oval;
        layers[m_layers_number-1]->neurons[n]->delta = 
                        oval * (1.0f - oval) * (dsrdvec[n] - oval);
    }

    //get deltas for hidden layers
    for (int l = m_layers_number - 2; l > 0; l--) {
        for (int n = 0; n < layers[l]->get_neurons_number(); n++) {
            delta = 0.0f;

            for (int i = 0; i < layers[l]->neurons[n]->get_output_links_number(); i++)
                delta += layers[l]->neurons[n]->outputs[i]->w * 
                         layers[l]->neurons[n]->outputs[i]->pinput_neuron->delta;

            oval = layers[l]->neurons[n]->oval;
            layers[l]->neurons[n]->delta = oval * (1 - oval) * delta;
        }
    }


    ////////correct weights for every layer///////////////////////////
    for (int l = 1; l < m_layers_number; l++) {
        for (int n = 0; n < layers[l]->get_neurons_number(); n++) {
            for (int i = 0; i < layers[l]->neurons[n]->get_input_links_number(); i++) {
                //dw = rule*Xin*delta + moment*dWprv
                dw = nrule * layers[l]->neurons[n]->inputs[i]->ival * 
                             layers[l]->neurons[n]->delta;       
                dw += alpha * layers[l]->neurons[n]->inputs[i]->dwprv;
                layers[l]->neurons[n]->inputs[i]->dwprv = dw;
                //correct weight
                layers[l]->neurons[n]->inputs[i]->w += dw;  
            }
        }
    }
}

Now, you may compose your own networks and proceed to typical classification tasks used in OCR, computer vision, and so on.

You must Sign In to use this message board.

FAQ  Noise Tolerance Very HighHighMediumLowVery Low  Search Messages    Layout Expand All MessagesThread ViewNo Javascript (slow)No Javascript Preview    Per page 102550

Msgs 1 to 22 of 22 (Total in Forum: 22) (Refresh) FirstPrevNext

help:my question feiyan2000000 5:14 13 Aug '08   thanks your file and demo.I have a question how I can get the data in pca.nn file .I am not find the dat file like virgi.dat and setosa.dat in the annldn ,waiting for you. Sign In·View Thread·PermaLink 1.00/5 (1 vote) Re: help:my question Chesnokov Yuriy 22:33 27 Aug '08   do PCA on them. See Matlab statistical toolbox for help chesnokov Sign In·View Thread·PermaLink my question feiyan2000000 23:21 12 Aug '08   thanks your file and demo.I have a question how I can get the data in pca.nn file .I am not find the dat file like virgi.dat and setosa.dat in the annldn ,waiting for you. Sign In·View Thread·PermaLink Question on class mark csille 1:23 14 Mar '08   Hello, I have a problem with class mark. In your exemple, you put only class 1 or 2 to define a positive(0.9 output desired) or negative(0.1 output desired) class. In my data training I don't have only positive or negative output. How can I put more output desired (different to true or false)? Is it possible ? Thank you very much for your help. Cecile Sign In·View Thread·PermaLink Re: Question on class mark Chesnokov Yuriy 21:31 18 Mar '08   Yes it is. Write main console function to handle multiple classes marks. ANN supports it. chesnokov Sign In·View Thread·PermaLink have a question [modified] gebitu 14:43 19 Feb '08   Hi,Chesnokov Yuriy, thanks your advises for me. If I need 3th output, how to design the nn config file? maybe I will set iris.nn file like : 3 4 8 2 and how to set trained data files? like: cls1.txt setosa1 1 50 33 14 2 cls2.txt virgi1 2 64 28 56 22 cls3.txt ??? test 3 10 63 10 20 and how to send consol command? ann1dn r cls1.txt cls2.txt 1000 ann1dn r cls1.txt cls3.txt 1000 ann1dn r cls2.txt cls3.txt 1000 ? Thks. modified on Tuesday, February 19, 2008 7:59 PM Sign In·View Thread·PermaLink 2.00/5 (1 vote) Re: have a question Chesnokov Yuriy 20:55 19 Feb '08   That console app on codeproject is limited to training on 1 output neuron only. My ANN lib supports any number of output neurons 3, 10, 100 ... You need just to modify main console file to feed to ANN lib desired outputs like: 0.9 0.1 0.1 for 1st class 0.1 0.9 0.1 for 2st class 0.1 0.1 0.9 for 3st class and so on, for any number of output neurons you have to put cls1, cls2, cls3, ... entries as you presented in your post to one file cls and provide just empty file like void and do the training: >>ann1dn.exe t net.nn cls void 1000 I've got developed before the same looking console that induces ANN to train and run on any number of outputs as I described, I can give you it but not for free chesnokov Sign In·View Thread·PermaLink 5.00/5 (1 vote) Re: have a question Chesnokov Yuriy 20:58 19 Feb '08   ... though may be I'll do update providing console for multiple outputs in future, but not sooner as I'm preoccupied a bit with coding chesnokov Sign In·View Thread·PermaLink Re: have a question gebitu 23:20 19 Feb '08   I send u a email at gmail.com. plz check it. Thks. Sign In·View Thread·PermaLink some ideas [modified] gebitu 22:31 17 Feb '08   Did u considered adding the incremental learning algorithm to ur program? The network have to retrain when the new data appear, If the amount of trained data is very large,ur program will take a long time to trained. Do u think the incremental learning algorithm could Speed up the network retraining? Could u give me some ideas about how to use this program on real-time motion analysis by a car cam ? Thk You. modified on Monday, February 18, 2008 3:56 AM Sign In·View Thread·PermaLink Re: some ideas Chesnokov Yuriy 3:27 18 Feb '08   That was the first alg I implemented, it took just some portion from train set, train on it, then select new subset with wrong classified data, train on it and so on, so on, until it recognizes all the train set. It took about an overnight run. Now on the same data see my face detection article, just 20 seconds on about 20000 vectors of 40 dimensions. Much faster than Matlab do. You can substitute floats for doubles, see my convolution article if you want even more speed. You needs terabytes of data and RAM to devise your training set to get days of training. If you encounter this, you've selected wrong features. It is not a good idea to collect millions of 640x480 bitmaps and do the training on that high dimensional data. It is the curse of dimensionality. You need compact features. Or wait for a quantum computers ANN is for pattern recognition, I do not know what kind of data you want to classify with motion analysis see my details at the bototm for a proposition. chesnokov Sign In·View Thread·PermaLink Re: some ideas gebitu 6:43 18 Feb '08   Thks your ans, but I have 3 more questions..... 1)what's mean the alg? 2)Do ur mean that u implemented incremental learning method in ur ann class? Just select new subset with wrong classified data, train on it and so on, dont change the struct of the ann ? 3)If I select a new vector and train on it, is this can be called incremental learning method? I will read ur all articles.Thanks. Sign In·View Thread·PermaLink Re: some ideas Chesnokov Yuriy 18:42 18 Feb '08   1) algorythm 2) yes. it is not in this code. just additional routine to run the ANN as in ann1Dn.cpp main console file. 3) probably. chesnokov Sign In·View Thread·PermaLink 5.00/5 (1 vote) hava a problem...... gebitu 17:45 15 Feb '08   I trained a nn and get a new file iris.nn , I copy red.dat and red.hea to the same dir with iris.nn. but why can't I use this command "ann1dn.exe r iris.nn red.dat" and cmd show this text "no files loaded from red.dat". Could u plz tell me,How to use the nn to cls red.dat? Thk You Sign In·View Thread·PermaLink Re: hava a problem...... Chesnokov Yuriy 1:53 17 Feb '08   See the article, it is the binary file format. >>ann1dn t network.nn data1_file data2_file 1000 The data1_file should contain full path to red.dat or the like files, and the same for data2_file: data1_file full path\red1.dat 1 full path\red2.dat 1 full path\red3.dat 1 data2_file full path\blue1.dat 2 full path\blue2.dat 2 full path\blue3.dat 2 You may also do not provide 1 and 2 class marks at the end of line of full path, if the class mark is absent each entry from data1_file will be of class 1 by default and class 2 from data2_file by default. You may also intermingle the file formats like: setosa1 1 50 33 14 2 setosa2 1 46 34 14 3 full path\setosa1.dat 1 full path\setosa2.dat 1 51 33 17 5 setosa5 1 virgi1 2 64 28 56 22 virgi2 2 67 31 56 24 virgi3 2 63 28 51 15 full path\virgi1.dat 1 full path\virgi2.dat 1 Remeber to provide your setosa and virgi dat files in binary format. chesnokov Sign In·View Thread·PermaLink Re: hava a problem...... gebitu 6:18 17 Feb '08   sorry, I should describe my problem detailed. I want to use my trained networks for testing the red.dat. so I just run the console with these parameters: F:\ann_demo\bin>ann1dn r iris.nn f:\ann_demo\bin\dat\red.dat loading data... no files loaded from: f:\ann_demo\bin\dat\red.dat. I got the above tips, I knew that the red.dat file is in the binary format. so,I just want to know how to use the trained networks to test the red.dat file. Thk You. Sign In·View Thread·PermaLink Re: hava a problem...... gebitu 7:05 17 Feb '08   I traced the source code and find this function as follows: int read_line(FILE *f, wchar_t *buff, int *c) { wint_t res = 0; ... return (short)res; } This function returns -1, so,ann1dn can't go on. The red.hea file was defined 10 3, so the red.dat file has 120 numbers, I find the above function read 121 times from the red.dat file, Last time it read the number 65535 and return -1. maybe is this a bug ? Sign In·View Thread·PermaLink Re: hava a problem...... Chesnokov Yuriy 7:13 17 Feb '08   There are no bugs in my code. red.dat contains 120 entries of 3 dimensional data. iris data set is 4 dimensional. You are going to train NN for iris data classification and test it on red color data??? You're not going to be trained for car driving and test your skills flying on the boeing chesnokov Sign In·View Thread·PermaLink Re: hava a problem...... Chesnokov Yuriy 7:06 17 Feb '08   It's all the same. Create a file test.txt or any other name and put there this line f:\ann_demo\bin\dat\red.dat or many more lines of full paths to your files, see my previous post. test.txt f:\ann_demo\bin\dat\red.dat f:\ann_demo\bin\dat\red1.dat f:\ann_demo\bin\dat\red2.dat if you do not know the class membership or test.txt f:\ann_demo\bin\dat\red.dat 1 f:\ann_demo\bin\dat\red1.dat 1 f:\ann_demo\bin\dat\red2.dat 1 f:\ann_demo\bin\dat\blue2.dat 2 f:\ann_demo\bin\dat\blue3.dat 2 if you know class membership and you will get Se,Sp,Pp,Np,Ac classification results. BWT. iris.nn is for iris data set 4 dimensional one. red.dat is 3 diminsional red color entries. They do not mate. You will get error like network dimensionality does not match input data dimensionality. chesnokov Sign In·View Thread·PermaLink 5.00/5 (1 vote) Re: hava a problem...... gebitu 21:21 17 Feb '08   Thks,