Click here to Skip to main content
14,870,673 members
Articles / Artificial Intelligence / Neural Networks
Article
Posted 7 Aug 2017

Stats

27.5K views
2K downloads
23 bookmarked

Basic Artificial Neural Network in C#

Rate me:
Please Sign up or sign in to vote.
4.94/5 (7 votes)
7 Aug 2017CPOL4 min read
A basic artificial neural network code for experimenting

Introduction

This article presents a basic Artificial Neural Network program in C#. The program offers a platform for experimenting on Artificial Neural Networks.

Background

The human brain can be viewed as a complex assemby of millions of neurons interconnected with each other. The information received by the brain is processed by these tiny neurons. Signals received by a neuron via dendrites and are further relayed by synapse. The synapses either pass a signal to adjoining neuron or not depending on the strength of the signal. As the human brain acquires information, new connections are developed between neurons and existing connections are modified.

Artificial Neural Network attempts to simulate (to a very basic level) the functioning of a human brain. A set of interconnected neurons are created. These interconnected neurons are organized in layers for easier handling. The signal transfer between neurons happen via dendrites. The potential of a neuron to "fire" a signal to the adjoining connected neuron is decided by an activation potential. Usually, this firing tendancy (also known as action potential) is modeled by a sigmoidal function. The sigmoidal function looks like:

$ S(x) = \frac{1}{1+e^{-x}} $

The sigmoidal function returns a value between 0 and 1 for any input of x. It is highly recommended that the reader studies through the properties of sigmoid function in order to appreciate its use as activation function. This article does not attempt to discuss the fundamentals of Artificial Neural Network. Instead, the readers are recommended to refer to more authorative texts on the subject.

Using the code

The Artificial Neural Network contains the following major classes:

Dendrite: This is the fundamental unit which transfers signal from one neuron to another. The signal transmitted through the dendrite is weighted to simulate the intensity of the signal being passed. It should be noted that this class exposes only one property - Weight.

C#
public class Dendrite
    {
        private double wt;
        public double Weight
        {
            get { return wt; }
            set { wt = value; }
        }

        //Provide a constructor for the class.
        //It is always better to provide a constructor instead of using
        //the compiler provided constructor
        public Dendrite()
        {
            wt = getRandom(0.00000001, 1.0);
        }

        private double getRandom(double MinValue, double MaxValue)
        {
            return Util.GetRandomD() * (MaxValue - MinValue) + MinValue;
        }

    }

Neuron: The neurons are the basic building blocks of the Artificial Neural Network. The neurons expose the value in it along with a bias. The neurons also contain properties representing the errors in the expected solutions. along with the most important parameter, the dendrites. Each neuron will have as many dendrites as the number of neurons in previous layer.

C#
class Neuron
    {
        private List<Dendrite> dendrites;
        private double bias, delta, _value;

        public double Bias
        {
            get
            { return bias; }
            set
            { bias = value; }
        }
        public double Delta
        {
            get
            { return delta; }
            set
            { delta = value; }
        }
        public double Value
        {
            get
            { return _value; }
            set
            { _value = value; }
        }

        public void AddDendrites(int nDendrites)
        {
            int i;
            //Dendrite d;
            for(i=0;i<nDendrites;i++)
            {
                //d = new Dendrite();
                dendrites.Add(new Dendrite());
            }
        }

        public int nDendrites()
        {
            return dendrites.Count;
        }

        public Dendrite getDendrite(int index)
        {
            return dendrites[index];
        }

        public Neuron()
        {
            bias = Util.GetRandomD();
            dendrites = new List<Dendrite>();
        }

    }

Layer: The layer class contains a list of neurons in the layer.

C#
class Layer
   {
       private List<Neuron> neurons;

       public void Clear()
       {
           neurons.Clear();
       }
       public void Initialize(int nNeurons)
       {
           int i;
           for(i=0;i<nNeurons;i++)
           {
               neurons.Add(new Neuron());
           }
       }

       public Neuron getNeuron(int index)
       {
           return neurons[index];
       }
       public void setNeuron(int index, ref Neuron neuron)
       {
           neurons[index] = neuron;
       }
       public void setNeuron(int index, Double value)
       {
           Neuron n = new Neuron();
           n.Value = value;
           neurons[index] = n;
       }

       public void AddDendritesToEachNeuron(int nDendrites)
       {
           int i;
           for(i=0;i<neurons.Count;i++)
           {
               neurons[i].AddDendrites(nDendrites);
           }
       }

       public int nNeurons()
       {
           return neurons.Count;
       }



       /// <summary>
       /// Constructor of the class
       /// </summary>
       public Layer()
       {
           neurons = new List<Neuron>();
       }
   }

Network: The network class contains a list of layers. Thus, the class hierarchy can be shown as:

Network -> Layer -> Neuron -> Dendrite

Once the network is initialized, the dendrites are assigned random weights. As these weights are random, the generated network is hardly of any use. The weights associated with each dendrite must be fine tuned in order to obtain meaningful results. This is known as training. A neural network requires to be trained for a given set of input and corresponding output data. The set of available data is known as Training Set. The training set usually forms approximately 80 - 85% of the total available data. The remaining available data is used to confirm the training. The training primarily involves determining the error at output node and distributing it in the form of change in dendrite weights. This method is known as Back Propagation. Back propagation is mathematically performed as gradient descent method. Readers are recommended to refer to a descriptive treatment of gradient descent method and is considered to be out of scope for this article.

  1. The training of a network is performed in the following way:
  2. Run the input
  3. Calculate the error on the output neurons
  4. From the error computed on output neurons, adjust the weights in dendrites. 
  5. Do this for all layers
  6. Repeat

The training may be terminated when the error norm reduces below threshold or may be terminated after a definite number of cycles. This code runs for a definite number of cycles.

Once the network is trained (and correctness confirmed), inputs with unknown outputs may be entered to obtain the output. Note that Artificial Neural Networks may NOT return 100% accurate results. The results are usually within acceptable accuracy limits.

Points of Interest

Various heuristic parameters such as learning rate requires significant experience before the the network may successfully be trained at a good speed. This code allows experimenting with number of hidden layers and number of neurons in each hidden layer. The code has been tested with sinusoidal function (between 0 and 1) and other functions such as addition, substraction, etc and has been found to work with significant accuracy.

History

This is the first version of the code.

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Share

About the Author

Bhairav Thakkar
Software Developer (Senior)
Canada Canada
No Biography provided

Comments and Discussions

 
Suggestion. Pin
Member 1447228419-Oct-19 19:48
MemberMember 1447228419-Oct-19 19:48 
QuestionGPU & EXCEL CSV Pin
Neo Tech14-Apr-19 4:39
MemberNeo Tech14-Apr-19 4:39 
QuestionIs there a bug in the Network class? Pin
Member 1203500822-Sep-18 3:27
MemberMember 1203500822-Sep-18 3:27 
QuestionThanks Pin
Member 1091697415-Sep-17 13:03
professionalMember 1091697415-Sep-17 13:03 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.