## Introduction

On September 30, 2019, Google announced that the final release of TensorFlow 2.0 (TF 2.0) is now available. From TensorFlow Guide, there are major changes in TF 2.0:

- Removing redundant APIs such as
`tf.app`

,`tf.flags`

,`tf.logging`

. - Executing eagerly like Python (Eager execution)
- Keeping track of your variables and if you lose track of a
`tf.Variable`

, TF 2.0 gets garbage collected. - Using
`tf.function`

.

You can discover more about new changes in TF 2.0 at TensorFlow Guide. In this post, I will represent some changes in TF 2.0 by creating linear regression models from scratch.

## Background

In one of my older articles, I introduced the linear regression algorithm and how to create a simple linear regression model using TensorFlow 1.X. In this post, I will update that model (with 2 weights) to TF 2.0 and I also create a model with more weights (of course, more than 2).

## Using the Code

### Simple Linear Regression Model (With Two Weights)

In TensorFlow 1.X, I created a model and implemented it with the following lines of code:

... # set up variable for weights w0 = tf.Variable(0.0, name="w0") w1 = tf.Variable(0.0, name="w1") ... # Define the operation that will be called on each iteration train_op = tf.train.GradientDescentOptimizer(learning_rate).minimize(costF) # set up a session sess = tf.Session() # initialize all variables init = tf.global_variables_initializer() # execute the session sess.run(init) # Loop through the data training for epoch in range(training_epochs): for (x, y) in zip(x_train, y_train): # execute the session sess.run(train_op, feed_dict={X: x, Y: y}) # get values of the final weights by executing the session w_val_0 = sess.run(w0) w_val_1 = sess.run(w1)

As code above, in TF 1.X, we must:

- declare variables (
`tf.Varialble`

) and initialize these variables (`tf.global_variables_initializer`

) before using them. - train our model using
`tf.train.GradientDescentOptimizer`

. - set up (
`tf.Session`

) and run the session to execute operations in the graph.

In TF 2.0, we will:

- declare variables (
`tf.Variable`

) but don't need to use`tf.global_variables_initializer`

, that means, TensorFlow 2.0 doesn’t make it mandatory to initialize variables. - train our model using tf.GradientTape and we will use assign_sub for weight variables.
- not require the session execution.

From the main points above, we will re-create our linear regression model with two weights as follows:

import tensorflow as tf import numpy as np import matplotlib.pyplot as plt learning_rate = 0.01 # steps of looping through all your data to update the parameters training_epochs = 100 # the training set x_train = np.linspace(0, 10, 100) y_train = x_train + np.random.normal(0,1,100) w0 = tf.Variable(0.) w1 = tf.Variable(0.) def h(x): y = w1*x + w0 return y def squared_error(y_pred, y_true): return tf.reduce_mean(tf.square(y_pred - y_true)) # train model for epoch in range(training_epochs): with tf.GradientTape() as tape: y_predicted = h(x_train) costF = squared_error(y_predicted, y_train) # get gradients gradients = tape.gradient(costF, [w1,w0]) # compute and adjust weights w1.assign_sub(gradients[0]*learning_rate) w0.assign_sub(gradients[1]*learning_rate) plt.scatter(x_train, y_train) # plot the best fit line plt.plot(x_train, h(x_train), 'r') plt.show()

If we run this script, the result can look like this:

### Polynomial Model

When data points appear to form smooth curves rather than straight lines, we need to change our regression model from a straight line to something else. One such approach is to use a polynomial model. A polynomial is a generalization of a linear function. The following lines of code will create a polynomial model:

import tensorflow as tf import numpy as np import matplotlib.pyplot as plt learning_rate = 0.05 training_epochs = 100 #the traning set x_train = np.linspace(-1, 1, 101) # Set up raw output data based on a degree 5 polynomial num_coeffs = 6 trY_coeffs = [1, 2, 3, 4, 5, 6] y_train = 0 for i in range(num_coeffs): y_train += trY_coeffs[i] * np.power(x_train, i) # Add some noise y_train += np.random.randn(*x_train.shape) * 1.5 # Set up the weight vector to all zeros w = tf.Variable([0.] * num_coeffs, name="parameters") # our model function def h(x): # h(x) = w5*x*x*x*x*x + w4*x*x*x*x + w3*x*x*x + w2*x*x + w1*x + w0 y = 0 for i in range(num_coeffs): y += w[i]*pow(x,i) return y # cost function def squared_error(y_pred, y_true): return tf.reduce_mean(tf.square(y_pred - y_true)) # train model for epoch in range(training_epochs): with tf.GradientTape() as tape: y_predicted = h(x_train) costF = squared_error(y_predicted, y_train) # get gradients gradients = tape.gradient(costF, w) # compute and adjust weights w.assign_sub(gradients*learning_rate) plt.scatter(x_train, y_train) # plot the best fit line plt.plot(x_train, h(x_train), 'r') plt.show()

If we run this script, the result can look like this:

Of course, we need to improve this model but I don't do that in this post.

## Points of Interest

TensorFlow is a great platform for deep learning and machine learning and TF 2.0 focuses on simplicity and ease of use. In this post, I introduced some new changes in TF 2.0 by building simple linear regression models from scratch (don't use APIs such as Keras) and I hope you feel excited about it.

## History

- 25
^{th}March, 2020: Initial version