Logo Recognition System






4.82/5 (10 votes)
Logo Recognition System Program written in C# .NET 6.0 Windows Form (Tensorflow.net,Tensorflow.keras,Emgu Cv,ScottPlot.WinForms,Newtonsoft.Json)
Introduction
In this article, I will present my program (Logo Recognition System) and show you the mechanism of the program by giving a step by step explanation of the code.
Note: To be able to understand the article, you should have general knowledge of the basics of deep learning, such as the components of an artificial neural network, such as the layer or the activation function you should also have advanced C# knowledge.
Before we get started, we need to install the required libraries. The following libraries are required to get our program working:
Tensorflow.net
andTensorflow.Keras
Using the Paket-Manager-Console, enter the following commands:
NuGet\Install-Package TensorFlow.NET -Version 0.100.2
NuGet\Install-Package TensorFlow.Keras -Version 0.10.2
NuGet\Install-Package SciSharp.TensorFlow.Redist -Version 2.10.0
NuGet\Install-Package SciSharp.TensorFlow.Redist-Windows-GPU -Version 2.10.0
- Emgu cv
Using the Paket-Manager-Console, enter the following commands:
NuGet\Install-Package EmguCV -Version 3.1.0.1
NuGet\Install-Package Emgu.CV.runtime.windows -Version 4.6.0.5131
NuGet\Install-Package Emgu.CV.Bitmap -Version 4.6.0.5131
ScottPlot.WinForms
Using the Paket-Manager-Console, enter the following command:
NuGet\Install-Package ScottPlot.WinForms -Version 4.1.60
- Newtonsoft.Json
Using the Paket-Manager-Console, enter the following command:
NuGet\Install-Package Newtonsoft.Json -Version 13.0.2
Note: After we installed the required libraries, we need also to set the target build platform from any CPU to 64 bit as the following:
Project->Properties->Build->General->Platform Target-> x64
Part 1: Create Our Own Convolution Neural Network Model
Since the convolution layer is specialized in image recognition and since we mainly want to classify images, we will use the convolution layer in our network and to speed up the learning process, we will also use the "maxpooling" layer and last but not least and to eliminate overfitting and to avoid a neural network being overly dependent on a particular object property, we will use the "droupout" layer in our network. As the final step in our cnn, we will use the "Dense" Layer which is a common classifier for neural networks. In this layer, every node is connected to every node in the previous layer. As activation function, we will use the Relu
function in the hidden layers to speed up the training and the softmax function in the output layer because we will have more than two outputs.
Step by Step Network Structure Implementation
- The network input is a 32 x 32 image, which means that the input size should be 32 x 32 with 3 RGB channels
- x2 convolutional layers with 32 filter and with filter size as 3x3
Note 1: To avoid reducing layer output size, we can set the padding property to "same". - maxpooling layer with size as 2x2
- droupout layer with 0.25 rate
Note 2: As a parameter, we can give this layer a value between 0 and 1, for example 0.25 means that when the network is trained, a predetermined number (25%) of neurons in a certain layer of the network will be switched off ("dropout") and become not processed for the next calculation step. - We repeat the 2, 3, 4 steps two times and in each time, we double the number of the filters count in the convolutional layer
- Dense layer with 256 neuron
- Dense layer with 128 neuron
- As output, we will also use the dense layer with softmax as activation function:
public class CNN
{
ILayer Conv2D_1_;
ILayer Conv2D_2_;
ILayer MaxPooling2D_1_;
ILayer Dropout_1_;
ILayer Conv2D_3_;
ILayer Conv2D_4_;
ILayer MaxPooling2D_2_;
ILayer Dropout_2_;
ILayer Conv2D_5_;
ILayer Conv2D_6_;
ILayer MaxPooling2D_3_;
ILayer Dropout_3_;
ILayer Flatten;
ILayer Dense_1_;
ILayer Dense_2_;
ILayer Dense_3_;
int outputCount;
public CNN(int outputCount) {
var layers = new LayersApi();
Conv2D_1_ = layers.Conv2D(32, (3, 3), activation: "relu", padding: "same");
Conv2D_2_ = layers.Conv2D(32, (3, 3), activation: "relu", padding: "same");
Conv2D_3_ = layers.Conv2D(64, (3, 3), activation: "relu", padding: "same");
Conv2D_4_ = layers.Conv2D(64, (3, 3), activation: "relu", padding: "same");
Conv2D_5_ = layers.Conv2D(128, (3, 3), activation: "relu", padding: "same");
Conv2D_6_ = layers.Conv2D(128, (3, 3), activation: "relu", padding: "same");
MaxPooling2D_1_ = layers.MaxPooling2D(pool_size: (2, 2));
MaxPooling2D_2_ = layers.MaxPooling2D(pool_size: (2, 2));
MaxPooling2D_3_ = layers.MaxPooling2D(pool_size: (2, 2));
Dropout_1_ = layers.Dropout(0.25f);
Dropout_2_ = layers.Dropout(0.25f);
Dropout_3_ = layers.Dropout(0.25f);
Flatten = layers.Flatten();
Dense_1_ = layers.Dense(256, activation: "relu");
Dense_2_ = layers.Dense(128, activation: "relu");
Dense_3_ = layers.Dense(outputCount, activation: "softmax");
}
public Model Build(string name)
{
var inputs = keras.Input(shape: (32, 32, 3), name: "img_"+name);
var x = Conv2D_1_.Apply(inputs);
x = Conv2D_2_.Apply(x);
x = MaxPooling2D_1_.Apply(x);
x = Dropout_1_.Apply(x);
x = Conv2D_3_.Apply(x);
x = Conv2D_4_.Apply(x);
x = MaxPooling2D_2_.Apply(x);
x = Dropout_2_.Apply(x);
x = Conv2D_5_.Apply(x);
x = Conv2D_6_.Apply(x);
x = MaxPooling2D_3_.Apply(x);
x = Dropout_3_.Apply(x);
x = Flatten.Apply(x);
x = Dense_1_.Apply(x);
x = Dense_2_.Apply(x);
return keras.Model(inputs, Dense_3_.Apply(x), name: name);
}
}
Part 2: The Program
To fully understand how to use the program, we need to know how it works in general so the program contains three parts; the first one where we prepare our own data set, the second part which represents the training process where we train our own model and the last part where we test our Model on realistic Objects.
-
Prepare Our Own Data Set
For this part, I have developed a Snipping Image Tool by which we can do two things:
-
The first thing is to Select an Object Manually where in this process, the Extracted Object will be exported as either a new Output and here a new Label in the Ideal Set Combo-box will be added or the Extracted Object can be set to existing Label in the Ideal Set Combobox, and that will be done as in the following steps in the order:
// Doses the Selected Object Already exists in the Data Set DialogResult res = CustomMsgbox.InputBox.ShowDialog ("represent the Image a new Dataset ?", "Question", CustomMsgbox.InputBox.Icon.Question, CustomMsgbox.InputBox.Buttons.YesNo); // if the Selected Object New or doesn´t exists in the Data Set, // we have to enter the new Ideal set Value if (res == System.Windows.Forms.DialogResult.Yes) { res = CustomMsgbox.InputBox.ShowDialog ("Please Enter the Ideal set value !!", "Ideal set", CustomMsgbox.InputBox.Icon.Question, CustomMsgbox.InputBox.Buttons.OkCancel, CustomMsgbox.InputBox.Type.TextBox, new string[] { "Ideal set new Value" }); if (res == System.Windows.Forms.DialogResult.OK) { Bitmap image = ImageHelper.CutImage(rect, new Bitmap (this.docu_pb.Image)); // Filter Selected Object Colors on the basis of Program Settings switch (Central_Static_Value.Settings.Filter) { case "Colored": image = ImageHelper.BGRFiler(image); break; case "Gray": image = ImageHelper.GrayFilter(image); break; case "BlackWeight": image = ImageHelper.BlackWhiteFilter(image); break; default: image = ImageHelper.BGRFiler(image); break; } // get Ideal Set Value from the Message Box string InputBox_ResultValue = CustomMsgbox.InputBox.ResultValue; // save the Ideal set Value as new Output int match = this.AssignIdentity(InputBox_ResultValue); //give the Image a Name to show in to Train Images dataGridView, //Note that this Name does not play any role //in the Actual training or Predict string imageName = "image" + Central_Static_Value.Train_Model. to_Train_Images_dataGridView.Rows.Count.ToString(); // add the Image in the to Train Images dataGridView Central_Static_Value.Train_Model. to_Train_Images_dataGridView.Rows.Add (imageName, InputBox_ResultValue, ImageHelper.ResizeImage(32, 32, image)); // Create a New Dataset Instance TrainingSet trainingSet = new TrainingSet(imageName, match, ImageHelper.ResizeImage(32, 32, image), true); // add the Dataset in the TrainingSetList Central_Static_Value.Train_Model.TrainingSetList.Add(trainingSet); // refresh the Ideal set Combobox refrech_ideal_set_comboBox(); } } // if the Selected Object already exists in the Data Set, // we have to choose one of the Available Ideal Set Values else { res = CustomMsgbox.InputBox.ShowDialog ("Please choose the Ideal set value !!", "Ideal set", CustomMsgbox.InputBox.Icon.Question, CustomMsgbox.InputBox.Buttons.OkCancel, CustomMsgbox.InputBox.Type.ComboBox, Central_Static_Value.Train_Model.neuron2identity.Select (x => x.Value).ToArray()); if (res == DialogResult.OK) { // get Ideal Set Value from the Message Box string result = CustomMsgbox.InputBox.ResultValue; //Cut the selected Area of the given Image Bitmap image = ImageHelper.CutImage (rect, new Bitmap(this.docu_pb.Image)); //give the Image a Name to show in to Train Images dataGridView, //Note that this Name does not play any role in //the Actual training or Predict string imageName = "image" + Central_Static_Value.Train_Model. to_Train_Images_dataGridView.Rows.Count.ToString(); // save the Ideal set Value as new Output int match = AssignIdentity(result); // Filter Selected Object Colors on the basis of Program Settings switch (Central_Static_Value.Settings.Filter) { case "Colored": image = ImageHelper.BGRFiler(image); break; case "Gray": image = ImageHelper.GrayFilter(image); break; case "BlackWeight": image = ImageHelper.BlackWhiteFilter(image); break; default: image = ImageHelper.BGRFiler(image); break; } // add the Image in the to Train Images dataGridView Central_Static_Value.Train_Model.to_Train_Images_ dataGridView.Rows.Add (imageName, result, ImageHelper.ResizeImage (32, 32, image)); // Create a New Dataset Instance TrainingSet trainingSet = new TrainingSet(imageName, match, ImageHelper.ResizeImage(32, 32, image), false); // add the Dataset in the TrainingSetList Central_Static_Value.Train_Model.TrainingSetList.Add(trainingSet); } }
-
After we use the Mouse, select a new Object. The program will ask us whether the selected object already exists in the dataset or not.
-
If the Selected Object is New or doesn't exist in the Data Set we have to enter the new Ideal set Value. Other case we have to choose one of the Available Ideal Set Values.
- After we entered the Ideal Set Value, we need to save it into a
Dictionary
where theKey
in theDictionary
represents the equivalent Current Output Count and the Value in theDictionary
represents the actual Ideal Set Value. Otherwise, we need to refresh the Output count Variable by incrementing it by one. All that, of course, if the Ideal set is New, refreshTrainingsetList
by adding the NewDataset
and the code: -
The second thing is to Select Objects automatically and that will be done using the Selective Search Algorithm) where in this process, the Extracted Object can be just set to existing Label in the Ideal Set Combo-box in the Snip Image Form so after we get one of them, using selective search automatic selected, object we take Ideal Set Value from the
ideal_set_cb
and save it as new Output.The code:
//get the Image Name ,to show it into Train Images dataGridView, //Note that this Name does not play any role //in the Actual training or Predict string imageName = Image_name_tb.Text; // get Ideal Set Value from the ideal_set_cb string selecteditem = ideal_set_cb.SelectedItem.ToString(); // save the Ideal set Value as new Output int match = AssignIdentity(selecteditem); // add the Image in the to Train Images dataGridView Central_Static_Value.Train_Model.to_Train_Images_dataGridView.Rows.Add (imageName, selecteditem, ImageHelper.ResizeImage (32, 32, new Bitmap(sniped_img_pictureBox.Image))); // Create a New Dataset Instance TrainingSet trainingSet = new TrainingSet (imageName, match, ImageHelper.ResizeImage (32, 32, new Bitmap(sniped_img_pictureBox.Image)), false); // add the Dataset in the TrainingSetList Central_Static_Value.Train_Model.TrainingSetList.Add(trainingSet);
-
-
-
Second Part "Training Process"
After the
Dataset
is prepared, the Training Process can be started as in the following steps:// Convert Trainigset Images List into Int 4 Dimensional ND Array NDArray x_train = ImageHelper.ImagesToNDArray(this.TrainingSetList, 32, 32); // Convert Trainigset Idealsets or Matches in Int 2 Dimensional ND Array NDArray y_train = ImageHelper.IdelaValuesToNDArray (this.TrainingSetList, this.outputCount); //Convert the Input Matrix Type Into float x_train = x_train.astype(np.float32); x_train /= 255; Model = new CNN(outputCount).Build("LogoDetector"); var opt = new Tensorflow.Keras.Optimizers.Adam(learning_rate: 0.001f); // Model.summary(); Model.compile(optimizer: opt, loss: keras.losses.CategoricalCrossentropy(), metrics: new[] { "acc" }); for (int i = 0; i < (int)Epoches_numericUpDown.Value; i++) { Model.fit(x_train, y_train); }
-
We already know that our network accepts a Size of (32x32) as Input so firstly and during the Prepare Dataset Process, we have to resize our image to (32x32) after that in the Training Process, we have to convert our images into
Int ND Array
so when we Convert our Image, we get an 1x32x32x3 Array, the first 1 represents the Image Index and the last 3 mean that we have 3 colors in each Pixel and now, let's imagine we have more than one image to train, so we need aND Array
like (Variable-Images-Count x 32 x 32 x 3) and the code:/// <summary> /// Convert List of Images into four dimensional ND Array where the /// first dímension represent the Image Index and the Second the /// Image width and the third the Imgae Height and the Last /// the Image Colors /// </summary> /// <param name="bitmaps">to be Converted Image List</param> /// <param name="width">Images Width</param> /// <param name="height">Images Height</param> /// <returns>4 Dimensional ND Array</returns> public static NDArray ImagesToNDArray(List<TrainingSet> bitmaps, int width, int height) { int[,,,] imagespixels = new int[bitmaps.Count, width, height, 3]; for (int bi = 0; bi < bitmaps.Count; bi++) { Bitmap bitmap = bitmaps[bi].Image; int[,,] pixels = getPixels(bitmap); for (int x = 0; x < width; x++) { for (int y = 0; y < height; y++) { for (int z = 0; z < 3; z++) { imagespixels[bi, x, y, z] = pixels[x, y, z]; } } } } return np.array(imagespixels); }
-
After we convert our Image into
ND Array
, we need also to convert our Ideal Dataset Values or the Matches Values into2d Nd Array
where the First Dimension represent the Equivalent Image Index and the second Dimension represent the Match value or the Ideal Output and the code:/// <summary> ///the Idealset or the right Predict represented in 1 ///for identical and 0 for not identical ///will here converted into two dimensional Array, ///where the first dimension represent the Image Index ///and the second the output or the Idealset ///</summary> /// <param name="images">the Ideal Sets of the Images List</param> /// <param name="outputCount">Count of the CNN Outputs</param> /// <returns>2 Dimensional ND Array</returns> public static NDArray IdelaValuesToNDArray (List<TrainingSet> images,int outputCount) { int[,] idealvalues = new int[images.Count, outputCount]; for (int i = 0; i < images.Count; i++) { for (int j = 0; j < outputCount; j++) { if (j == images[i].Match) idealvalues[i, j] = 1; else idealvalues[i, j] = 0; } } return np.array(idealvalues); }
-
To improve our training result, we divide our data set or our 4D dimensional
ND Array
by255
, the result will be a matrix with values between 0 and 1. -
To speed up our learning process and save memory requirements, we will use the adam optimizer.
-
After that, the learning process can be started, and the final code:
-
-
Part 3, "Testing Process"
-
The program needs firstly a selective search algorithm, this algorithm will select for us all object in the Image and prepare them for the next step namely to Predict what for object it is:
-
The Predict Process: In this step, the Program will tell us what for object it is in the Image for that the Program will convert the image into 3 dimensional ND Array, the first dimension represent the width and the second the height and the last the RGB or the Colors. After that, we will give the array to our CNN to process it and at last to know what for object it is.
-
Note
In case we try to Predict images from outside the range of the dataset on which the model has been trained, there is a possibility that false predictions will be generated. Here, I would like to point out that the model must be trained on all images that have been set through the selective search algorithm, because only in this case will the program return correct predictions.
In the following video, I will show you how to use the program:
History
- 19th February, 2023: Initial version
- 28th March, 2023: Article updated