I spent a lot of time converting Lasagne's code to TensorFlow Keras. Although I successfully ran the code, however, I could not obtain the result reported in the paper.
Some help is available on
python - convert Lasagne to Keras code (CNN -> LSTM) - Stack Overflow, but it is limited to specific layers and not the entire architecture.
Lasagne Cod
net = {}
net['input'] = lasagne.layers.InputLayer((BATCH_SIZE, 1, SLIDING_WINDOW_LENGTH, NB_SENSOR_CHANNELS))
net['conv1/5x1'] = lasagne.layers.Conv2DLayer(net['input'], NUM_FILTERS, (FILTER_SIZE, 1))
net['conv2/5x1'] = lasagne.layers.Conv2DLayer(net['conv1/5x1'], NUM_FILTERS, (FILTER_SIZE, 1))
net['conv3/5x1'] = lasagne.layers.Conv2DLayer(net['conv2/5x1'], NUM_FILTERS, (FILTER_SIZE, 1))
net['conv4/5x1'] = lasagne.layers.Conv2DLayer(net['conv3/5x1'], NUM_FILTERS, (FILTER_SIZE, 1))
net['shuff'] = lasagne.layers.DimshuffleLayer(net['conv4/5x1'], (0, 2, 1, 3))
net['lstm1'] = lasagne.layers.LSTMLayer(net['shuff'], NUM_UNITS_LSTM)
net['lstm2'] = lasagne.layers.LSTMLayer(net['lstm1'], NUM_UNITS_LSTM)
net['shp1'] = lasagne.layers.ReshapeLayer(net['lstm2'], (-1, NUM_UNITS_LSTM))
net['prob'] = lasagne.layers.DenseLayer(net['shp1'],NUM_CLASSES, nonlinearity=lasagne.nonlinearities.softmax)
net['shp2'] = lasagne.layers.ReshapeLayer(net['prob'], (BATCH_SIZE, FINAL_SEQUENCE_LENGTH, NUM_CLASSES))
net['output'] = lasagne.layers.SliceLayer(net['shp2'], -1, 1)
Problem
I believe that all of the lines I converted are correct, except lst two lines of Lasagne code, which are as follows:
net['shp2'] = lasagne.layers.ReshapeLayer(net['prob'], (BATCH_SIZE, FINAL_SEQUENCE_LENGTH, NUM_CLASSES))
net['output'] = lasagne.layers.SliceLayer(net['shp2'], -1, 1)
I'm not sure how to convert these lines in Tensorflow Keras after my
output_layer, or what these lines are for.
What I have tried:
I come up with this in TensorFlow Keras
def CNN_model(input_shape, total_classes):
input_layer = tf.keras.Input(shape=input_shape, name="Time_Series_Activity")
con_l1 = tf.keras.layers.Conv2D(64, (5, 1), activation="relu", data_format='channels_first')(input_layer)
con_l2 = tf.keras.layers.Conv2D(64, (5, 1), activation="relu", data_format='channels_first')(con_l1)
con_l3 = tf.keras.layers.Conv2D(64, (5, 1), activation="relu", data_format='channels_first')(con_l2)
con_l4 = tf.keras.layers.Conv2D(64, (5, 1), activation="relu", data_format='channels_first')(con_l3)
permute_layer = tf.keras.layers.Permute((2, 1, 3))(con_l4)
rl = Reshape((int(permute_layer.shape[1]), int(permute_layer.shape[2]) * int(permute_layer.shape[3])))(permute_layer)
lstm_l5 = tf.keras.layers.LSTM(128, return_sequences=True, dropout=0.5)(rl)
lstm_l6 = tf.keras.layers.LSTM(128, dropout=0.5)(lstm_l5)
output_layer = tf.keras.layers.Dense(total_classes, activation="softmax")(lstm_l6)
return tf.keras.models.Model(inputs=input_layer, outputs=output_layer)