Click here to Skip to main content
Click here to Skip to main content
Go to top

Neural Dot Net Pt 2 The Basic Classes

, 22 Oct 2003
Rate this:
Please Sign up or sign in to vote.
A neural network library in C#.

Introduction

Like any serious program these days, there is going to be a class hierarchy in the background hiding all the fiddly stuff so that the main classes can just get on with doing the things they are designed to do, and this collection of neural network classes is no different. All these classes are based on the class library provided in the book Objected Orientated Neural Networks in C++, by Joey Rogers. These classes have been completely rewritten in C# although the basic functionality of the originals has been retained for the base classes. The most noticeable change in the code is through the enforcement of stronger type checking in C#, so whereas the line array[ i ]->Function is perfectly valid in C++, this is incorrect syntax in C# and should be written as ( ( Type )array[ i ] ).Function. I should also point out that there have been changes to the class structure in the translation from C++ to C#. Some classes have had only minor changes while some have been dropped altogether. Further changes are in the pipeline, but for now, it is better to keep things as simple as possible. The following is meant only as an overview of the class structure with the details of what is going on in the classes being referred to in part three where we will look at an actual implementation of these classes.

Sample image

The Values Class

The Values class is a representation of what where global variables in the original framework. The Values class holds four integer members that are used for accessing the arrays at specific points in the code. It is declared as a static member of the Basic class which is its one and only instance and it can be accessed through the inheritance chain by all other classes and even indirectly in the implementation code. The only function that the Values class implements is the Random function which is a static function so that it can be accessed easily from anywhere within the code. This function was moved from the Basic Node class as it's a singular function by nature and should be off to the side somewhere rather than being accessed through one of the main base classes.

The Basic Class

The Basic class serves as the base class for most of the Basic classes and simply contains a way to access the values and the identifier. In theory, there is no reason why the Basic and the Values class couldn't be a single class. The class is abstract as there should be no reason for anyone wanting to create an object of this class.

The BasicLink Class

The BasicLink class is the class that provides the glue between the nodes and contains the link value that is so important in the calculations made later on. The class contains an array of link values which are the weights for the links. There was certainly some confusion about this when I was learning about this stuff, but whenever anyone talks of weights in neural nets, this is where you will find them. The array for the most part will contain a single value, but as it is using an array list, it can contain as are required. You can even keep a track of all the previous weights that were used for this link if you so wish. The BasicLink class is declared abstract so that the implementer of a network is forced to inherit from this class. This is due to the fact that I wished to make the basic classes all uninstantiable, although I gave up on having this idea as a rule with the basic node class, as I felt that the requirement to derive from it was pointless, unless you were going to add to the functionality of the class and the BasicNode class as it is perfectly capable of acting as an input class for the Adaline and other networks. The link class' main purpose as well as providing the actual link is implement the weight functionality that consists of getting and setting the weights and the error values associated with the links and updating the weight values with the new value. When the run function is called and the code gets the weight, if the answer is incorrect, then part of the learning process means that the weight for each link is updated with the update function in an attempt to get the correct answer at the next epoch or loop through the data.

Important Bits

The important functions within the link classes are:

public virtual double GetLinkValue( int nID )
{

    if( debugLevel.TestDebugLevel( DebugLevelSet.Progress ) == true )
    {
        log.Log( DebugLevelSet.Progress, 
            "Getting the Link value for array number " 
             + Identifier.ToString() 
             + " link id no " + ID.ToString() + " value = " 
             + ( ( double )arrayLinkValues[ nID ] ).ToString(), ClassName );
    }

    if( arrayLinkValues.Count == 0 || nID > arrayLinkValues.Count )
        return 0.0;

    return ( double )arrayLinkValues[ nID ];
}

public virtual void SetLinkValue( double dNewValue, int nID )
{
    if( debugLevel.TestDebugLevel( DebugLevelSet.Progress ) == true )
    {
        log.Log( DebugLevelSet.Progress, 
            "Setting the link value at " + Identifier.ToString() + " from " 
            + ( ( double )arrayLinkValues[ nID ] ).ToString() + " to " 
            + dNewValue.ToString() + " link id no " 
            + ID.ToString(), ClassName );
    }

    if( arrayLinkValues.Count == 0 || nID > arrayLinkValues.Count )
    {
        if( debugLevel.TestDebugLevel( DebugLevelSet.Errors ) == true )
        {
            log.Log( DebugLevelSet.Warning, 
                "Error the id " + Identifier.ToString() 
                + " is greater than the number of link values or the 
                link values array is empty, link id no "
                + ID.ToString(), ClassName );
        }

        return;
    }

    arrayLinkValues[ nID ] = dNewValue;
}

The GetLinkValue and SetLinkValue are functions for accessing the links value, each function prints a message to the log marking its progress and checks that the accessor value to the array represented by the integer nID is within the bounds of the array that it is trying to access.

The link class also has access to all the values that it shares between the input and the output nodes.

public virtual double InputValue( int nID )
{
        if( debugLevel.TestDebugLevel( DebugLevelSet.Progress ) == true )
        {
            log.Log( DebugLevelSet.Progress, 
                "Getting the Input Value at " + Identifier.ToString() 
                + " link id no " + ID.ToString() + " value = "
                + InputNode.GetValue( nID ).ToString(), ClassName );
        }

        return InputNode.GetValue( nID );
}

/// 
/// get the value of the output node at the given id
/// 
/// 
/// 
public virtual double OutputValue( int nID )
{
    if( debugLevel.TestDebugLevel( DebugLevelSet.Progress ) == true )
    {
        log.Log( DebugLevelSet.Progress, 
            "Getting the Output Valuer at " + Identifier.ToString() 
            + " link id no " + ID.ToString() + " value = " 
            + OutputNode.GetValue( nID ).ToString(), ClassName );
    }

    return OutputNode.GetValue( nID );
}

/// 
/// get the value of the input error at the given id
/// 
public virtual double InputError( int nID )
{
    if( debugLevel.TestDebugLevel( DebugLevelSet.Progress ) == true )
    {
        log.Log( DebugLevelSet.Progress, 
            "Getting the Input Error at " + Identifier.ToString()  
            + " link id no " 
            + ID.ToString() + " value = " 
            + ( ( double )InputNode.GetError( nID ) ).ToString(), ClassName );
    }

    return ( double )InputNode.GetError( nID );
}

/// 
/// get the value of the output error at the given id
/// 
/// 
/// 
public virtual double OutputError( int nID )
{
        if( debugLevel.TestDebugLevel( DebugLevelSet.Progress ) == true )
        {
            log.Log( DebugLevelSet.Progress, 
                "Getting the Output Error at " + Identifier.ToString()
                + " link id no " + ID.ToString() + " value = " 
                + OutputNode.GetError( nID ).ToString(), ClassName );
        }

        return OutputNode.GetError( nID );
}

As you can see from the four accessor functions above, the link can get the data only from both the input and the output node that it maintains the link between. No error checking is done within the link class on these functions as that would duplicate the error handling contained within the classes themselves. However, apart from providing the glue between the input and the output nodes, the link's main function is to control the weights between them.

///
/// get the weighted input value at the given id
/// 
/// 
/// 
public virtual double WeightedInputValue( int nID )
{
    double dReturn = 0.0;  
    if(    Values.Weight <  arrayLinkValues.Count )
        dReturn = bnInputNode.GetValue( nID ) 
           * ( ( double )arrayLinkValues[ Values.Weight ] );
    else
    {
        if( debugLevel.TestDebugLevel( DebugLevelSet.Warning ) == true )
        {
            log.Log( DebugLevelSet.Warning, 
                "Warning the Values weight value is greater than the 
                link values count returning 0.0, 
                link id no " + ID.ToString(), ClassName );
        }
    }

    if( debugLevel.TestDebugLevel( DebugLevelSet.Progress ) == true )
    {
        log.Log( DebugLevelSet.Progress, 
            "Getting the Weighted Input Value at " + Identifier.ToString() 
            + " link id no " + ID.ToString() 
            + " unweighted value = " 
            + bnInputNode.GetValue( nID ).ToString() 
            + " weighted value = " + dReturn.ToString(), ClassName );
    }

    return dReturn;
}

/// 
/// get the weighted output value at the given id
/// 
/// 
/// 
public virtual double WeightedOutputValue( int nID )
{
    double dReturn = 0.0;  
    if( Values.Weight <  arrayLinkValues.Count )
        dReturn = bnOutputNode.GetValue( nID ) 
            * ( ( double )arrayLinkValues[ Values.Weight ] );
    else
    {
        if( debugLevel.TestDebugLevel( DebugLevelSet.Warning ) == true )
        {
            log.Log( DebugLevelSet.Warning, 
                "Warning the Values Weight value is greater that the 
                link values array count returning 0.0, link id no " 
                + ID.ToString(), ClassName  );
        }
    }

    if( debugLevel.TestDebugLevel( DebugLevelSet.Progress ) == true )
    {
        log.Log( DebugLevelSet.Progress, 
            "Getting the Weighted Output value at " 
            + Identifier.ToString() + " link id no " 
            + ID.ToString() + " unweighted value = " 
            + bnOutputNode.GetValue( nID ).ToString() 
            + " weighted value = " + dReturn.ToString(), ClassName );
    }

    return dReturn;
}

/// 
/// get the weighted output error at the given id
/// 
/// 
/// 
public virtual double WeightedOutputError( int nID )
{
    double dReturn = 0.0;  
    if(    Values.Weight <  arrayLinkValues.Count )
        dReturn = bnOutputNode.GetError( nID ) 
            * ( ( double )arrayLinkValues[ Values.Weight ] );
    else
    {
        if( debugLevel.TestDebugLevel( DebugLevelSet.Warning ) == true )
        {
            log.Log( DebugLevelSet.Warning, 
                "Warning the Values Weight value is greater that the 
                link values array count returning 0.0, link id no " 
                + Identifier.ToString(), ClassName );
        }
    }

    if( debugLevel.TestDebugLevel( DebugLevelSet.Progress ) == true )
    {
        log.Log( DebugLevelSet.Progress, 
            "Getting the Weighted Output Error value at " 
            + Identifier.ToString() + " link id no " + Identifier.ToString()
            + " unweighted value = " 
            + bnOutputNode.GetError( nID ).ToString() + " weighted value = " 
            + dReturn.ToString(), ClassName );
    }

    return dReturn;
}

/// 
/// get the weighted input error at the given id
/// 
/// 
/// 
public virtual double WeightedInputError( int nID )
{
    double dReturn = 0.0;  
    if(    Values.Weight <  arrayLinkValues.Count )
        dReturn = bnInputNode.GetError( nID ) 
               * ( ( double )arrayLinkValues[ Values.Weight ] );
    else
    {
        if( debugLevel.TestDebugLevel( DebugLevelSet.Warning ) == true )
        {
            log.Log( DebugLevelSet.Warning, 
                "Warning the Values Weight value is greater than the 
                link values array count returning 0.0, link id no " 
                + Identifier.ToString(), ClassName );
        }
    }

    if( debugLevel.TestDebugLevel( DebugLevelSet.Progress ) == true )
    {
        log.Log( DebugLevelSet.Progress, 
            "Getting the Weighted Input Error value at " 
            + Identifier.ToString() + " link id no " + Identifier.ToString() 
            + " unweighted value = " + bnInputNode.GetError( nID ).ToString()
            + " weighted value = " + dReturn.ToString(), ClassName );
    }

    return dReturn;
}

/// 
///  Update the weight for this basic Link
/// 
/// 
public virtual void UpdateWeight( double dNewValue )
{
    if( Values.Weight < arrayLinkValues.Count )
    {
        double dTemp = ( double )arrayLinkValues[ Values.Weight ];
        dTemp += dNewValue;

        if( debugLevel.TestDebugLevel( DebugLevelSet.Progress ) == true )
        {
            log.Log( DebugLevelSet.Progress, 
                "Updating the Weight for this basic link id no " 
                + Identifier.ToString() 
                + " from = " 
                + ( ( double )arrayLinkValues[ Values.Weight ]).ToString() 
                + " to = " + dTemp.ToString(), ClassName );
        }

        arrayLinkValues[ Values.Weight ] = dTemp;

    }
}

There are four main accessor functions that the link controls with regard to the weights, with these allowing the weighted values for the input value, the output value, the input error and the output error values all being returned with their original values multiplied by the weight value of the current link. The final UpdateWeight function is called when the learn function is activated and the weight value for the link is required to be updated by the output node.

The BasicNode Class

The BasicNode class is the workhorse of the application and is the only Basic class that is not abstract at this point. This is due to the fact that it is used to provide input classes for the Adaline network at least. In the original C++ class library, there was another class that derived from this but it didn't really add any functionality to the BasicNode class, so I removed it and allowed this class to be instantiable.

The BasicNode class keeps arrays of the values where array 0 is usually the node value and array 1 is usually the learning rate, although it should be noted that this is usually for derived classes only and that when the Basic Node is being used as an Input Node, then it will only contain one value in the values array 0. It also contains the errors that have been generated on this node at Node Errors array 0. The node also contains ArrayLists to both the inputs and the outputs of the nodes, which will need to be lists as we will see later when we start to look at nodes that have many input connects. The BasicNode class is the class responsible for implementing the connection functions that create the links between the input nodes and the output nodes which take the form of connect this node to the node passed to the function using the links passed as the second parameter to the function.

Important Bits

The three most important functions for the basic node class are:

public virtual void Run( int nMode )
{
    if( debugLevel.TestDebugLevel( DebugLevelSet.Progress ) == true )
    {
        log.Log( DebugLevelSet.Progress, 
            "Run called in basic node", ClassName );
    }
}

public virtual void Learn( int nMode )
{
    if( debugLevel.TestDebugLevel( DebugLevelSet.Progress ) == true )
    {
        log.Log( DebugLevelSet.Progress, 
            "Learn Called in basic node", ClassName );
    }
}

public virtual void Epoch( int nMode )
{
    if( debugLevel.TestDebugLevel( DebugLevelSet.Progress ) == true )
    {
        log.Log( DebugLevelSet.Progress, 
            "Epoch called in basic node", ClassName );
    }

}

These functions are for implementing the main functionality of the Nodes. The Run function is the function that will implement the networks algorithm for that node. The Learn function will update the weight value in the link and the Epoch function in theory controls the entire run through the network although this is more a concept for the BasicNetwork class or the BasicNeuron class. All three functions are designed to be implemented in base classes and are not used in the BasicNode class which makes the BasicNode class ideal for use as an input class, as the fact that these functions are not implemented means that the user wont become confused and break things by inadvertently calling any of them on an input node. The BasicNode class also implements some functions for the getting and setting of the values in the four arrays that it holds:

private ArrayList arrayNodeValues; /// double values
private ArrayList arrayNodeErrors; /// double values
private ArrayList arrayInputLinks; /// Basic links
private ArrayList arrayOutputLinks; /// Basic links

These arrays hold all the information that the BasicNode requires. The Node Values array holds the value(s) for the node and the Node Errors array holds the error value(s) for the node. Each node also keeps track of the input and output links to the node, although when used as an input node, only this class will only have output nodes. The connections between the nodes are controlled by:

///
/// create a link to a node
/// 
/// 
/// 
public void CreateLink( BasicNode bnToNode, BasicLink blLink )
{
    if( debugLevel.TestDebugLevel( DebugLevelSet.Progress ) == true )
    {
        log.Log( DebugLevelSet.Progress, 
            "Creating a link between node " + bnToNode.Identifier.ToString() 
            + " and link " 
            + blLink.Identifier.ToString(), ClassName );
    }

    arrayOutputLinks.Add( blLink );
    bnToNode.InputLinks.Add( blLink );
    blLink.InputNode = this;
    blLink.OutputNode = bnToNode;
}

/// 
/// disconnect a connection from the arrays
/// 
/// 
/// 
/// true on success
public bool Disconnect( BasicNode bnFromNode, BasicNode bnToNode )
{
    if( debugLevel.TestDebugLevel( DebugLevelSet.Progress ) == true )
    {
        log.Log( DebugLevelSet.Progress, 
            "Disconnecting the connection between " 
            + bnFromNode.ID.ToString()
            + " to " + bnToNode.ID.ToString(), ClassName );
    }

    ArrayList tempList = bnFromNode.OutputLinks;
    bool bFound = false;

    /// see if it is possible to find the connection to the to node
    int n=0;
    for( ; n<tempList.Count; n++ )
    {
        if( ( ( BasicLink )tempList[ n ] ).OutputNode.Equals( bnToNode ) 
               == true )
        {
            bFound = true;
            break;
        }
    }

    if( bFound == true )
    {
        /// from the current node remove the input then remove the node
        ( ( BasicLink )tempList[ n ] ).OutputNode.arrayInputLinks.Remove( 
                 tempList[ n ] );
        tempList.RemoveAt( n );
        return true;
    }
    else
        return false;
}

The CreateLink function is called on the node and sets up the link to the passed in node through the passed in link.

The BasicNetwork Class

The BasicNetwork class is a container class that holds the information for the whole network. This version of the network class has no inherent structure in that it simply holds arrays of all the nodes and the links and is designed to make the holding of a number of nodes and links easier than having a large collection held within the program. For networks such as the Adaline network, this class is able to provide constructors that will build the whole network. For larger networks, it has the functionality to add nodes and links. There are some thoughts at the moment to develop a network class that uses the BasicNeuron class but this has been put off for now in order to concentrate on the actual neural network programming rather than getting bogged down in implementation details. The class is an abstract class and needs to be derived from in order to see it. It does, however, provide the basic accessing functionality that would be required by any class that wants to derive from this class. It also has a fully implemented save function and contains the blanked out code for loading the file. A true implementation of load is not possible at this level as it would require an instance of the BasicLink class which cannot be instantiated as it is an abstract class. The code is provided as an example and there is a working example in the Adaline network code. The class also declares an abstract CreateNetwork function which is used to build the network, an example of this will be given in the Adaline network.

Important Bits

public virtual void AddNode( BasicNode node )
{
    if( debugLevel.TestDebugLevel( DebugLevelSet.Progress ) == true )
    {
        log.Log( DebugLevelSet.Progress, 
            "Adding a new neuron to basic network" );
    }

    arrayNodes.Add( node );
}

public virtual void AddLink( BasicLink link )
{
    if( debugLevel.TestDebugLevel( DebugLevelSet.Progress ) == true )
    {
        log.Log( DebugLevelSet.Progress, 
            "Adding a new link to the basic network" );
    }

    arrayLinks.Add( link );
}

public virtual void RemoveNodeAt( int nID )
{
    if( debugLevel.TestDebugLevel( DebugLevelSet.Progress ) == true )
    {
        log.Log( DebugLevelSet.Progress, 
            "Removing a node from basic network at " + nID.ToString() );
    }

    if( nID <= arrayNodes.Count )
    {
        arrayNodes.RemoveAt( nID );
    }
    else
    {
        if( debugLevel.TestDebugLevel( DebugLevelSet.WarningsAndErrors ) 
             == true )
        {
            log.Log( DebugLevelSet.WarningsAndErrors, 
                "Warning attempt to remove a node from basic network " 
                + nID.ToString() 
                + " when there are only " 
                + arrayNodes.Count.ToString() + " in the array " );
        }
    }
}

public virtual void RemoveLinkAt( int nID )
{
    if( debugLevel.TestDebugLevel( DebugLevelSet.Progress ) == true )
    {
        log.Log( DebugLevelSet.Progress, 
            "Removing a link from the basic network at " + nID.ToString() );
    }

    if( nID <= arrayNodes.Count )
    {
        arrayNodes.RemoveAt( nID );
    }
    else
    {
        if( debugLevel.TestDebugLevel( DebugLevelSet.WarningsAndErrors ) 
                == true )
        {
            log.Log( DebugLevelSet.WarningsAndErrors, 
                "Warning attempt to remove a node from basic network " 
                 + nID.ToString() 
                 + " when there are only " + arrayLinks.Count.ToString() 
                 + " in the array " );
        }
    }
}

public BasicNode GetNodeAt( int nID )
{
    if( arrayNodes.Count >= nID )
        return ( BasicNode )arrayNodes[ nID ];
    else
    {
        if( debugLevel.TestDebugLevel( DebugLevelSet.WarningsAndErrors )
             == true )
        {
            log.Log( DebugLevelSet.WarningsAndErrors,
                "Warning attempt to get a node from basic network " 
                + nID.ToString() + " when there are only " 
                + arrayNodes.Count.ToString() + " in the array " );
        }
    }

    return null;
}

public BasicLink GetLinkAt( int nID )
{
    if( arrayLinks.Count >= nID )
        return ( BasicLink )arrayLinks[ nID ];
    else
    {
        if( debugLevel.TestDebugLevel( DebugLevelSet.WarningsAndErrors ) 
                == true )
        {
            log.Log( DebugLevelSet.WarningsAndErrors, 
                "Warning attempt to get a link from basic network " 
                + nID.ToString() + " when there are only " 
                + arrayLinks.Count.ToString() + " in the array " );
        }
    }

    return null;
}

The BiasNode Class

The BiasNode is a simple child of the BasicNode class that has its value set to 1.0 by default. Although this class is present in the implementation of the Adaline network, later it is not actually used by any of the code, as the Adaline relies on the values provided by the learning rate, the node value and the error.

The BasicNeuron Class

Sample image

The BasicNeuron class is not a part of the original framework but I thought that it made more sense to have a class that encapsulated the idea of a neuron as an object. I think this makes the design conceptually easier in that it separates things into more clearly defined separate layers; in that you now have network that contains neurons and these neurons contain nodes that do the work, rather than having a network that has a whole bunch of nodes that have no clearly defined relationship, unless you understand the program already.

The BasicNeuron contains the essentials for building a neuron, these being a couple of input nodes and a bias node, as well as an ArrayList for holding the links to the nodes. This class is also declared as abstract to force the implementer to inherit from it. This is a bit more reasonable than the abstract declaration of the link class, as the Neuron class for a specific network will normally want to add some functionality to this class, whereas I suspect most implementations of BasicLink derived classes will simply be calling the base class functionality.

The Pattern Class

Sample image

The Pattern class is used for loading the information that is presented to the network. This class will hold the values in its own arrays and present them to the network a pair at a time.

Finally

Right then, that is the introduction to the basic classes. Now, let's move on and look at how the Adaline 1 program actually works, and as part of that, look into what these classes do in action.

History

  • 24 June 2003 :- Initial release.
  • 23 October 2003 :- Review and edit for CP conformance.

References

  • Tom Archer (2001) Inside C#, Microsoft Press
  • Jeffery Richter (2002) Applied Microsoft .NET Framework Programming, Microsoft Press.
  • Charles Peltzold (2002) Programming Microsoft Windows With C#, Microsoft Press
  • Robinson et al (2001) Professional C#, Wrox
  • William R. Staneck (1997) Web Publishing Unleashed Professional Reference Edition, Sams.net
  • Robert Callan, The Essence Of Neural Networks (1999) Prentice Hall
  • Timothy Masters, Practical Neural Network Recipes In C++ (1993) Morgan Kaufmann (Academic Press)
  • Melanie Mitchell, An Introduction To Genetic Algorithms (1999) MIT Press
  • Joey Rogers, Object-Orientated Neural Networks in C++ (1997) Academic Press
  • Simon Haykin, Neural Networks A Comprehensive Foundation (1999) Prentice Hall
  • Bernd Oestereich (2002) Developing Software With UML Object-Orientated Analysis And Design In Practice, Addison Wesley
  • R Beale & T Jackson (1990) Neural Computing An Introduction, Institute Of Physics Publishing

Thanks

Special thanks go to anyone involved in TortoiseCVS for version control.

All UML diagrams were generated using Metamill version 2.2.

License

This article has no explicit license attached to it but may contain usage terms in the article text or the download files themselves. If in doubt please contact the author via the discussion board below.

A list of licenses authors might use can be found here

Share

About the Author

pseudonym67

United Kingdom United Kingdom
No Biography provided

Comments and Discussions

 
QuestionLinks?? PinmemberNiklas Ulvinge26-May-05 5:50 
GeneralO please! Pinmemberleppie24-Jun-03 7:49 
GeneralRe: O please! PineditorBarry Lapthorn24-Jun-03 8:54 
GeneralRe: O please! PinsussAnonymous26-Jun-03 2:15 
GeneralRe: O please! PinsussAnonymous26-Jun-03 2:16 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

| Advertise | Privacy | Mobile
Web04 | 2.8.140921.1 | Last Updated 23 Oct 2003
Article Copyright 2003 by pseudonym67
Everything else Copyright © CodeProject, 1999-2014
Terms of Service
Layout: fixed | fluid