Click here to Skip to main content
15,860,943 members
Articles / Multimedia / GDI+

Large pattern recognition system using multi neural networks

Rate me:
Please Sign up or sign in to vote.
4.94/5 (71 votes)
31 May 2012CPOL7 min read 210.8K   253.2K   192   45
Tutorials of using multi neural networks for large pattern recognition system, handwriting recognition system

<object type="application/x-shockwave-flash" codebase="http://download.macromedia.com/pub/shockwave/cabs/flash/swflash.cab#version=3,0,0,0" width="480" height="360" data="http://www.youtube.com/v/5OxLUI8CSoE?version=3&hl=vi_VN"><param name="movie" value="http://www.youtube.com/v/5OxLUI8CSoE?version=3&hl=vi_VN" /><param name="quality" value="high" /><param name="wmode" value="transparent" />

Introduction 

Now a day, artificial neural network has been applied popularly in many fields of human life. However, creating an efficient network for a large classifier like handwriting recognition systems is still a big challenge to scientists. In my last article named “Library for online handwriting recognition system using UNIPEN database”, I presented an efficient library for a handwriting recognition system which can create, change a neural network simply. The demo program showed good recognition results to digit set (97%) and alphabet sets (93%).This article I will continue to present a solution for a large patterns classification in general and handwriting recognition in particular.

<o:p>

<o:p>

 Image 1

Image 2

Recognition rate significantly increate when using additional spell checker module 

Neural network for a recognition system  

In the traditional model of pattern recognition, a hand-designed feature extractor gathers relevant information from input and eliminates irrelevant variabilities. A trainer classifier (normally, a standard, fully-connected multi-layer neural network can be used as a classifier) then categorizes the resulting feature vectors into classes. However, it could have some problems which should influent to the recognition results. The convolution neural network (CNN) solves this shortcoming of traditional one to achieve the best performance on pattern recognition task. <o:p>

The CNNs is a special form of multi-layer neural network. Like other networks, CNNs are trained by back propagation algorithms. The difference is inside their architecture. The convolutional network combines three architectural ideas to ensure some degree of shift, scale, and distortion invariance: local receptive field, shared weights (or weight replication) spatial or temporal sub-sampling. They have been designed especially to recognize patterns directly from digital images with the minimum of pre-processing operations. The architecture details of CNN have been described comprehensively in articles of Dr. Yahn LeCun and Dr. Patrice Simard (see my previous articles).

 Image 3

Figure 1: The Architecture of LeNET 5

<o:p>

Image 4

<o:p>

Figure 2: An input image followed by a feature map performing a 5 × 5 convolution and a 2 x 2 sub-sampling map 

The recognition results of the above networks are really high to small patterns collection such as digit, capital letters or lower case letters etc. However, when we want to create a larger neural network which can recognize a bigger collection like digit and English letters (62 characters) for example, the problems begin appear. Finding an optimized and large enough network becomes more difficult, training network by large input patterns takes much longer time. Convergent speech of the network is slower and especially, the accuracy rate is significant decrease because bigger bad written characters, many similar and confusable characters etc. Furthermore, assuming we can create a good enough network which can recognize accurately English characters but it certainly cannot recognize properly a special character outsize its outputs set (a Russian or Chinese character) because it does not have expansion capacity. Therefore, creating a unique network for very large patterns classifier is very difficult and may be impossible.

The proposed solution to the above problems is instead of using a unique big network we can use multi smaller networks which have very high recognition rate to these own output sets. Beside the official output sets (digit, letters…) these networks have an additional unknown output (unknown character). It means that if the input pattern is not recognized as a character of official outputs it will be understand as an unknown character. Then the input pattern will be transferred to the next network until the system can recognize it correctly. 

<o:p>

 Image 5

Figure 3: Convolution neural network with unknown output

<o:p>

 Image 6

Figure 4: Recognition System using multi neural networks

This solution overcomes almost limits of the traditional model. The new system includes a several small networks which are simple for optimizing to get the best recognition results. Training these small networks takes less time than a huge network. Especially, the new model is really flexible and expandable. Depending on the requirement we can load one or more networks; we can also add new networks to the system to recognize new patterns without change or rebuilt the model. All these small networks have reusable  capacity to an other multi neural networks system. 

Experiment 

The demo program is built to the purpose showing all stages of a recognition system including: create a component network, train a network, test networks on UNIPEN dataset and test networks on a mouse drawing control. It is tutorials which can help everybody can understand to a recognition system. All functions can be implemented on the program GUI. So you can create, train, and test your network on runtime without change any code or restart the program.<o:p>

 Image 7

<o:p>

Figure 5: Handwriting recognition system interface 

Creating new neural network 

Image 8

Figure 6: Creating new neural network Interface

Creating new neural network completely bases on GUI. Creating a network depends on the input pattern size, number of layers, data set…. On the output layer we can choose unknown output checkbox to create an additional unknown output to the network or ignore it to create a normal network.<o:p>

<o:p>

Of course, we can still to create a network by code:

C#
         void CreateNetwork()
<pre>        {
            network = new ConvolutionNetwork();
            //layer 0: inputlayer
            network.Layers = new Layer[6];
            network.LayerCount = 6;
            InputLayer inputlayer = new InputLayer("00-Layer Input", new Size(29, 29));
            network.InputDesignedPatternSize = new Size(29, 29);
            inputlayer.Initialize();
            network.Layers[0] = inputlayer;
            ConvolutionLayer convlayer = new ConvolutionLayer("01-Layer ConvolutionalSubsampling", inputlayer, new Size(13, 13), 10, 5);
            convlayer.Initialize();
            network.Layers[1] = convlayer;
            convlayer = new ConvolutionLayer("02-Layer ConvolutionalSubsampling", convlayer, new Size(5, 5), 60, 5);
            convlayer.Initialize();
            network.Layers[2] = convlayer;
            FullConnectedLayer fulllayer = new FullConnectedLayer("03-Layer FullConnected", convlayer, 200);
            fulllayer.Initialize();
            network.Layers[3] = fulllayer;
            fulllayer = new FullConnectedLayer("04-Layer FullConnected", fulllayer, 100);
            fulllayer.Initialize();
            network.Layers[4] = fulllayer;
            OutputLayer outputlayer = new OutputLayer("05-Layer Output", fulllayer, Letters3.Count, true);
            outputlayer.Initialize();
            network.Layers[5] = outputlayer;
            network.TagetOutputs = Letters3;
            network.UnknownOuput = '?';
        }  

Training a network

 After creating a neural network using "Create network" function, the network will be trained using UNIPEN database.

 Image 9

Figure 7: Training network interface

<o:p>

<o:p>

Depending on the network size we can choose training set is 1a, 1b or 1c in the UNIPENdata folder. Statistic of training process can show many useful information such as: No. of epoch, MSE, training time per epoch, success rate…<o:p>

UNIPEN data browser and recognition testing<o:p>

The UNIPEN data browser control in the demo program can show all the UNIPEN data files. We can also test the trained neural network on these files by loading trained network parameters files.Image 10 

Figure 8: UNIPEN data browser and recognition interface

<o:p>

 Mouse Drawing test

 Image 11

Figure 9: Mouse drawing recognition interface

<o:p>

The mouse drawing control is based on the excellent article ”DrawTools” by Alex Fr.  I just changed some codes to fit to my requirement. The cursive text in the image is divided to line, word and isolated character by same algorithm as follows: 

    private void btRecognition_Click(object sender, EventArgs e)
<pre>        {
            //recognition all characters in the drawArea
            if (bitmap != null)
            {
                bitmap.Dispose();
                bitmap = null;
            }
            bitmap = new Bitmap(drawArea.Width, drawArea.Height);
            drawArea.DrawToBitmap(bitmap, new Rectangle(0, 0, bitmap.Width, bitmap.Height));
            drawBitmap =(Bitmap) bitmap.Clone();
            if (bitmap != null)
            {
                lbRecognizedText.Items.Clear();
                List<InputPattern> lineList=null;
                List<InputPattern> wordList=null;  
                InputPattern parentPt=new InputPattern(bitmap,255,new Rectangle(0,0,bitmap.Width,bitmap.Height));
                lineList = GetPatternsFromBitmap(parentPt,500,1,true,10,10);
                if (lineList.Count > 0)
                {
                        
                    if (characterList != null)
                    {
                        characterList.Clear();
                        characterList = null;
                    }
                    characterList = new List<InputPattern>();
                    foreach (var line in lineList)
                    {
                        String text = "";
                        wordList = GetPatternsFromBitmap(line, 50, 10,false, 10, 10);
                        if (wordList != null)
                        {
                            if (wordList.Count > 0)
                            {
                                foreach (var word in wordList)
                                {
                                    List<InputPattern> charList = GetPatternsFromBitmap(word, 5, 5, false, 10, 10);
                                    //check if have part bitmaps
                                    if (charList != null)
                                    {
                                        if (charList.Count > 0)
                                        {
                                            panelNavigation.Visible = true;
                                            foreach (var c in charList)
                                            {
                                                characterList.Add(c);
                                                c.GetPatternBoundaries(5,5,false,10,10);
                                                Char accChar = new Char();
                                                PatternRecognition(c.OriginalBmp,out accChar);
                                                if (accChar != '\0')
                                                {
                                                    text = String.Format("{0}{1}", text, accChar.ToString());
                                                    drawBitmap = c.DrawChildPatternBoundaries(drawBitmap);
                                                }
                                            }
                                        }
                                    }
                                    text = String.Format("{0} ", text);
                                }
                                  
                            }
                        }
                        lbRecognizedText.Items.Add(text);
                    }
                }
                pbPreview.Image = drawBitmap;
                lblNavigation.Text = characterList.Count.ToString();
                index = 0;
            }
            
        }  

 Image 12

Figure 10: Loading trained network parameters files 

In order to active the recognition function I simply load trained network parameters files. Depending to my recognition requirement I can load one, two or all files. The recognition results are really good (higher 90%) if I load only one network to recognize its output characters. However, when I load multi network the system’s accuracy rate becomes lower. The main reasons are many confusable characters in cursive text; the training sets are not large enough etc. 

For a large pattern collection like handwritten characters, there are so many similar characters which can make not only machine but also human confuse in some cases such as: O, 0 and o; 9, 4,g,q etc. These characters can make networks misrecognize. Hence the solution has been being upgraded which significant increate recognition rate by using an additional spellchecker/voting module at the output of system. The input pattern will be recognized by all component networks. These outputs (except unknown outputs) then will be set as the inputs of the spellchecker/voting module. The module will bases on previous recognized characters, internal dictionary and other factors to decide which one will be the most accurated recognized character.

Image 13

Figure 11: The new recognition system using Spell checker /voting module

Image 14

The new recognition system using Spell checker /voting module (internal dictionary)

Image 15

The spellchecker module makes the system recognizes much better  

Conclusion 

The proposed recognition model has solved amost prolems to a large recognition system: the capacity of recognizing large partern collection, flexible design and deployment, expanable and resuable capacity...etc. Increasing accuracy rate to the system also can do easier by increasing recognition rate of component networks, using the spell checker /voting module etc. The demo program also proved the capacity of the library which should be used in many other applications such as prediction application, face recognition... 

Fututre work and upgrade 

Some features would be udate to the library:

- Convolution and sampling layer of LeNET model.

- Spell checker / voting module 

-character segmentation. 

At the moment, the project took to much my free time. It should be slowdown or temporary stop until I can re-arrange everything and/or find a new good sponsorship. Howerver the vote/comment to the article would decice the project will continue or not. I will really appreciate to receive  comments and suggessions to the article especially to the model, spell checker module and character segmentation algorithm...

 History

version 1.0: initial code

version 1.1 the spell checker /voting module has been added to the system which increates significantly recognition rate. It made me really supprised and happied. I will publish it when I complete code rearrangement. 

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)


Written By
Vietnam Maritime University
Vietnam Vietnam
This member has not yet provided a Biography. Assume it's interesting and varied, and probably something to do with programming.

Comments and Discussions

 
QuestionNice article, with one suggestion Pin
Vimvq198729-May-12 20:15
Vimvq198729-May-12 20:15 
AnswerRe: Nice artical, with one suggestion Pin
Vietdungiitb29-May-12 21:00
Vietdungiitb29-May-12 21:00 
GeneralRe: Nice artical, with one suggestion Pin
Vimvq198729-May-12 21:07
Vimvq198729-May-12 21:07 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.