How to Save CNTK Model to File in C#

The final process of training is the model which should be used in the production as safe, reliable and accurate. Usually, model training is a frustrating and time consuming process, and it is not like we can see as demo to introduce with the library. Once the model is built with the right combination of parameter values and network architecture, the process of modeling turns into a interesting and funny task, since it calculates the values just as we expect.

In most of the time, we have to save the current state of the model, and continue with training due to various reasons:

• to change the parameters of the learner
• to switch from one to another machine
• or to share the state of the model with your team mate
• to switch from CPU to GPU
• etc.

In all the above cases, the current state of the model should be saved, and continue the training process from the last stage of the model. Because, the training from the beginning is not a solution, since we already invest time and knowledge to achieve progress of the model building.

The CNTK supports two kinds of persisting the model.

• production/ready or evaluation ready state, and
• saving the checkpoint of the model for later training

In the first case, the model is prepared for the evaluation and production but cannot be trained again, because it is freed from all other information but for the evaluation. During the saving process, only one file is generated.

In the second case, beside a model file, another file is generated with the name “modelname.ckp”. The file contains all information needed for the continuation of training. Once the trainer checkpoint is persisted, we can continue with model training even if we changed the following:

• the training data set with the same dimensions and data types
• the parameters of the learner
• the learner

What we cannot change in order to continue with training is the network model. In other words, the model must remain with the same number of layers, input and output dimensions.

Saving, Loading and Evaluating the Model

Once the model is trained, it can be persisted as separated file. As separate file, it can be loaded and evaluated with different dataset, but the number of the features and the label must remain the same as in case when was trained. Use this method when you want to share the model with someone else, or when you want to deploy the model in the production.

The model is saved simply by calling the CNTK method Save:

public void SaveTrainedModel(Function model, string fileName)
{
    model.Save(fileName);
}

The model evaluation requires several steps:

• load the model from the file
• extract the features and label from the model
• call evaluate method from the model, by passing the batch created from the features, label and the evaluation dataset

The model is loaded by calling Load method.

public Function LoadTrainedModel(string fileName, DeviceDescriptor device)
{
   return Function.Load(fileName, device, ModelFormat.CNTKv2);
}

Once the model is loaded, features and label are extracted from the model in the following way:

//load the model from file
Function model = Function.Load(modelFile, device);
//extract features and label from the model
Variable feature = ffnn_model.Arguments[0];
Variable label = ffnn_model.Output;

The next step is creating the minibatch in order to pass the data to the evaluation. In this case, we are going to create only one row for the Iris example of:

//Example: 5.0f, 3.5f, 1.3f, 0.3f, setosa
float[] xVal = new float[4] { 5.0f, 3.5f, 1.3f, 0.3f };
Value xValues = Value.CreateBatch<float>(new int[] {feature.Shape[0] }, xVal, device);
//Value yValues = - we don't need it, because we are going to calculate it

Once we created the variable and values, we can map them and pass to the model evaluation, and calculate the result:

//map the variables and values
var inputDataMap = new Dictionary<Variable, Value>();
inputDataMap.Add(feature,xValues);
var outputDataMap = new Dictionary<Variable, Value>();
outputDataMap.Add(label, null);
//evaluate the model
ffnn_model.Evaluate(inputDataMap, outputDataMap, device);
//extract the result  as one hot vector
var outputData = outputDataMap[label].GetDenseData<float>(label);

The evaluation result should be transformed to proper format, and compared with the expected result:

//transforms into class value
var actualLabels = outputData.Select(l => l.IndexOf(l.Max())).ToList();
var flower = actualLabels.FirstOrDefault();
var strFlower = flower == 0 ? "setosa" : flower == 1 ? "versicolor" : "virginica";
Console.WriteLine($"Model Prediction: 
                  Input({xVal[0]},{xVal[1]},{xVal[2]},{xVal[3]}), Iris Flower={strFlower}");
Console.WriteLine($"Model Expectation: 
                  Input({xVal[0]},{xVal[1]},{xVal[2]},{xVal[3]}), Iris Flower= setosa");

Training Previous Saved Model

Training previously saved model is very simple, since it requires no special coding. Right after the trainer is created with all necessary stuff (network, learning rate, momentum and other), you just need to call:

 trainer.RestoreFromCheckpoint(strIrisFilePath);

No additional code should be added.
The above method is called after you successfully saved the model state by calling:

trainer.SaveCheckpoint(strIrisFilePath);

The method is usually called at the end of the training process.
The complete source code from this blog post can be found here.

Filed under: .NET, C#, CNTK, CodeProject
Tagged: .NET, C#, CNTK, Code Project, CodeProject, Deep Neural Network, Machine Learning