Click here to Skip to main content
Click here to Skip to main content

Neural Dot Net Pt 10 The Self Organizing Word Network

By , 30 Nov 2003
Rate this:
Please Sign up or sign in to vote.

Introduction

As with the previous word versions of the networks, this one is merely an extension of the code that has gone before except perhaps more so. This network could have been written without any extra classes being added at all. In fact, the Self Organizing Network Word Pattern class that was written for this example is never used as I reuse classes from the Back Propagation network.

I wont go into detail about the classes here as they add nothing that hasn't been said before. If anyone wishes to use them, they are provided in their own file. However, for the sake of completeness, I will go through the details of how the network is used here, so that if anyone wants to look it up, there isn't a sudden break with the way the rest of the code is documented.

The Self Organizing Word Network

The Self Organizing Word Network basically implements the same functionality as the Self Organizing Network.

For details on implementation, see the base class descriptions above.

Training

The training loop for the Self Organizing Word Network consists of two loops, one for the training and one for the test. The additional loop that was used for the Self Organizing Network has been omitted from the Word version as I didn't feel it was required.

for( nIteration=0; nIteration<nNumberOfSonTwoIterations; nIteration++ )
{
    for( int i=0; i<patterns.Count; i++ )
    {
        for( int n=0; n<20; n++ )
        {
            /// Note because I use the LoadBackPropagationWord
            /// training file function the 
            ///pattern should be cast to an adaline word pattern which 
            /// contains the required function to get
            /// the values from the words.
            ( ( BasicNode )sonTest.Nodes[ n ] ).SetValue( 
               Values.NodeValue, 
               ( ( AdalineWordPattern )patterns[ i ] ).GetInSetAt( n ) );    
        }

        soNetwork.Run();
        netWorkText.AppendText( "." );
    }

    soNetwork.Learn();

    soNetwork.Epoch();

    log.Log( DebugLevelSet.Progress, "Iteration number " 
           + nIteration.ToString() + " produced a winning node at  " 
           + soNetwork.WinningHorizontalPos + " Horizontal and " 
           + soNetwork.WinningVerticalPos + " vertical, winning node value = " 
           + soNetwork.GetWinningNodeValue( soNetwork.WinningHorizontalPos, 
           soNetwork.WinningVerticalPos ) + "\n", ClassName );
    netWorkText.AppendText( "\nIteration number " 
           + nIteration.ToString() + " produced a winning node at  " 
           + soNetwork.WinningHorizontalPos + " Horizontal and " 
           + soNetwork.WinningVerticalPos 
           + " vertical, winning node value = " 
           + soNetwork.GetWinningNodeValue( soNetwork.WinningHorizontalPos, 
           soNetwork.WinningVerticalPos ) + "\n" );
}

netWorkText.AppendText( "Saving the network\n" );

FileStream xmlstream = new FileStream( "selforganizingnetworktwo.xml",
     FileMode.Create, FileAccess.Write, FileShare.ReadWrite, 8, true );
XmlWriter xmlWriter = new XmlTextWriter( xmlstream, System.Text.Encoding.UTF8 );
xmlWriter.WriteStartDocument();

soNetwork.Save( xmlWriter );

xmlWriter.WriteEndDocument();
xmlWriter.Close();

/// now load the file
FileStream readStream = new FileStream( "selforganizingnetworktwo.xml", 
     FileMode.Open, FileAccess.Read, FileShare.ReadWrite, 8, true );
XmlReader xmlReader = new XmlTextReader( readStream );

netWorkText.AppendText( "Loading the network\n" );
SelfOrganizingNetworkWordNetwork sonTest = 
        new SelfOrganizingNetworkWordNetwork( log );

sonTest.Load( xmlReader );
xmlReader.Close();

StringBuilder strDataDisplay = new StringBuilder( "" );
ArrayList arrayOutput = new ArrayList();

SelfOrganizingNetworkData data;
netWorkText.AppendText( "Completed the test ... 
   now reprocessing the orginal data through the loaded network\n " ); 

for( int i=0; i<patterns.Count; i++ )
{
    for( int n=0; n<20; n++ )
    {
        /// Note because I use the LoadBackPropagationWord
        /// training file function the 
        /// pattern should be cast to an adaline word pattern which 
        /// contains the required function to get the values from the words.
        ( ( BasicNode )sonTest.Nodes[ n ] ).SetValue( Values.NodeValue,
              ( ( AdalineWordPattern )patterns[ i ] ).GetInSetAt( n ) );    
    }

    sonTest.Run();

    strDataDisplay.Remove( 0, strDataDisplay.Length );
    strDataDisplay.Append( "Run Called at " + i.ToString() 
      + " Network Values are :- Composite Value = " 
      + sonTest.GetPosition( Values.Composite ) 
      + ", Horizontal Value = " + sonTest.GetPosition( Values.Row ) 
      + ", Vertical Value = " + sonTest.GetPosition( Values.Column ) 
      + ", Current Winning Horizontal Position = " 
      + sonTest.WinningHorizontalPos 
      + ", Current Winning Vertical Position " 
      + sonTest.WinningVerticalPos + ", Inputs = " 
      + ((BasicNode)sonTest.Nodes[0]).NodeValues[Values.NodeValue].ToString() 
      + "," 
      + ((BasicNode)sonTest.Nodes[1]).NodeValues[Values.NodeValue].ToString() 
      + ", Winning Node Value = " 
      + sonTest.GetWinningNodeValue( sonTest.WinningHorizontalPos, 
      sonTest.WinningVerticalPos ) + "\n" );

    strDataDisplay.Append( " String Data :- " );

    for( int n=0; n<( ( AdalineWordPattern )patterns[ i ] ).InputSize(); n++ )
    {
        strDataDisplay.Append( 
          ( ( AdalineWordPattern )patterns[ i ] ).InputValue( n ) + " " );
    }

    netWorkText.AppendText( strDataDisplay.ToString() ); 

    data = new SelfOrganizingNetworkData();
    data.CompositeValue = ( int )sonTest.GetPosition( Values.Composite );
    data.Data = strDataDisplay.ToString();

    arrayOutput.Add( data );
}

The first loop performs the training of the network for the required number of iterations, which in this example is five hundred. The code then loads the patterns value for the words into the NodeValue section of the node. Note, I use the AdalineWord pattern for this as well as reuse some of the earlier functions for loading the file. This is done for this demo to reduce the amount of cut and paste code that is used, although if you were developing an application using the Self Organizing Network Word files then you would need to write this yourself.

The Learn and Epoch functions are then called at the end of each iteration. The Learn function updates the nodes surrounding the winning node the area of which is controlled by the neighborhood size variable.

The code then does what by now should be standard saving and reloading of the network and then reruns it through an identical loop without calling the Learn or the Epoch functions, with the results of the output using the same formatting code as the Self organizing Network.

Saving And Loading

Saving and Loading is done using the same XML format as everywhere else and looks like:

<?xml version="1.0" encoding="utf-8"?>
    <SelfOrganizingNetworkWordNetwork>
        <SelfOrganizingNetwork>
            <HorizontalSize>10</HorizontalSize>
            <VerticalSize>10</VerticalSize>
            <InitialLearningRate>0.5</InitialLearningRate>
            <LearningRate>0.01</LearningRate>
            <FinalLearningRate>0.01</FinalLearningRate>
            <InitialNeighborhoodSize>5</InitialNeighborhoodSize>
            <FinalNeighborhoodSize>1</FinalNeighborhoodSize>
            <NeighborhoodDecrement>100</NeighborhoodDecrement>
            <NeighborhoodSize>1</NeighborhoodSize>
            <NumberOfIterations>500</NumberOfIterations>
            <Iterations>500</Iterations>
            <WinningHorizontalPosition>7</WinningHorizontalPosition>
            <WinningVerticalPosition>0</WinningVerticalPosition>
            <NumberOfNodes>20</NumberOfNodes>
            <InputLayer>
                <BasicNode>
                    <Identifier>0</Identifier>
                    <NodeValue>0.1074</NodeValue>
                    <NodeError>0</NodeError>
                    <Bias>
                        <BiasValue>1</BiasValue>
                    </Bias>
                </BasicNode>

                through to ...

                <BasicNode>
                    <Identifier>19</Identifier>
                    <NodeValue>0.2406</NodeValue>
                    <NodeError>0</NodeError>
                    <Bias>
                        <BiasValue>1</BiasValue>
                    </Bias>
                </BasicNode>
            </InputLayer>
            <KohonenLayer>
                <SelfOrganizingNetworkNode>
                    <BasicNode>
                        <Identifier>20</Identifier>
                        <NodeValue>36.7925330599203</NodeValue>
                        <NodeValue>0.01</NodeValue>
                        <NodeError>0</NodeError>
                        <Bias>
                            <BiasValue>1</BiasValue>
                        </Bias>
                    </BasicNode>
                </SelfOrganizingNetworkNode>
                
                through to ....
                
                <SelfOrganizingNetworkNode>
                    <BasicNode>
                        <Identifier>2099</Identifier>
                        <NodeValue>36.793609556095</NodeValue>
                        <NodeValue>0.01</NodeValue>
                        <NodeError>0</NodeError>
                        <Bias>
                            <BiasValue>1</BiasValue>
                        </Bias>
                    </BasicNode>
                </SelfOrganizingNetworkNode>
                <SelfOrganizingNetworkLink>
                    <BasicLink>
                        <Identifier23</Identifier>
                        <LinkValue>8.38539366591963</LinkValue>
                        <InputNodeID>2</InputNodeID>
                        <OutputNodeID>20</OutputNodeID>
                    </BasicLink>
                </SelfOrganizingNetworkLink>
                
                through to ....

                <SelfOrganizingNetworkLink>
                    <BasicLink>
                        <Identifier>2119</Identifier>
                        <LinkValue>8.72511583748183</LinkValue>
                        <InputNodeID>19</InputNodeID>
                        <OutputNodeID>2099</OutputNodeID>
                    </BasicLink>
                </SelfOrganizingNetworkLink>
            </KohonenLayer>
        </SelfOrganizingNetwork>
    </SelfOrganizingNetworkWordNetwork>

Testing

The testing portions of the code are located under the Run menu for the Neural Net Tester program. The test for this program is the "Load And Run Self Organizing Network 2" menu option. This will load the file that resembles the one above. I say resembles as the linkage values wont be exactly the same any two times running.

The menu option will load and run the SelfOrganizingNetworkOne.wrk file and generate the log Load And Run Self Organizing Network One.xml which can be viewed using the LogViewer that is part of the neural net tester program.

The display will show an output similar to that found when running the Adaline networks and is described in understanding the output below.

The quick guide is:

  • Menu :- Run/Load And Run Self Organizing Network 2:- Loads the saved Self Organizing network from the disk and then runs it against the origin-of-the-species.txt file.
  • Menu :- Train/Self Organizing Network 2 :- Trains the network from scratch using the sample file which by default is originpart.txt.
  • Menu :- Options/Self Organizing Network 2 Options :- Brings up a dialog that allows you to set certain parameters for the running of the network.

Options

As with the previous Self organizing Network, the Learning Rate is reduced as the program progresses. This is why the first two options are the starting or initial Learning Rate and the Final Rate. The Neighborhood which is the range of updated nodes when a Learn is called is reduced as the program progresses as well as the learning rate.

The neighborhood decrement is the number of iterations to perform before a reduction in the neighborhood size and the Number Of Iterations is the amount of times that you want the program to run through the training loop.

Fun And Games

The main problem with the output of this code (well, my main problem anyway) is just what exactly does it mean? The code is meant as an experiment just to see what turns up although actually interpreting the answers then becomes a problem in itself that raises another and perhaps the most important question of: Do the answers given by the network tell us anything about the book and the words in the book or do they merely tell us something about the nature of the words used in a mathematical sense? I personally can't decide on this one although I have a deep suspicion that the answer will be more to do with the nature of the numbers used, which when you get down to it are chosen in a perfectly arbitrary fashion. But then again, what if the technique can be used to tell us something about the book itself and the nature of how the book is written and if it works on this book, what about other books. It could possibly be quite fascinating, it could equally possibly be a complete waste of time. The problem brings me back to once again interpreting the data in a meaningful way.

At present, I don't have any answers to this at all. It's something that requires more research, and possibly, a mathematician having a go at it before a sensible answer is found.

History

  • 7 July 2003 :- Initial release.
  • 1 December 2003 :- Review and edit for CP conformance.

References

  • Tom Archer (2001) Inside C#, Microsoft Press
  • Jeffery Richter (2002) Applied Microsoft .NET Framework Programming, Microsoft Press
  • Charles Peltzold (2002) Programming Microsoft Windows With C#, Microsoft Press
  • Robinson et al (2001) Professional C#, Wrox
  • William R. Staneck (1997) Web Publishing Unleashed Professional Reference Edition, Sams.net
  • Robert Callan, The Essence Of Neural Networks (1999) Prentice Hall
  • Timothy Masters, Practical Neural Network Recipes In C++ (1993) Morgan Kaufmann (Academic Press)
  • Melanie Mitchell, An Introduction To Genetic Algorithms (1999) MIT Press
  • Joey Rogers, Object-Orientated Neural Networks in C++ (1997) Academic Press
  • Simon Haykin Neural Networks A Comprehensive Foundation (1999) Prentice Hall
  • Bernd Oestereich (2002) Developing Software With UML Object-Orientated Analysis And Design In Practice, Addison Wesley
  • R Beale & T Jackson (1990) Neural Computing An Introduction, Institute Of Physics Publishing.

Thanks

Special thanks go to anyone involved in TortoiseCVS for version control.

All UML diagrams were generated using Metamill version 2.2.

License

This article has no explicit license attached to it but may contain usage terms in the article text or the download files themselves. If in doubt please contact the author via the discussion board below.

A list of licenses authors might use can be found here

About the Author

pseudonym67

United Kingdom United Kingdom
No Biography provided

Comments and Discussions

 
QuestionWhat it really does PinmemberAlois Kraus3-Dec-03 7:44 
GeneralWell done! Pinmemberxxxyyyzzz1-Dec-03 4:34 
GeneralRe: Well done! Pinmemberpseudonym671-Dec-03 4:55 
GeneralRe: Well done! PinmemberST1-Dec-03 7:25 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

| Advertise | Privacy | Mobile
Web01 | 2.8.140421.2 | Last Updated 1 Dec 2003
Article Copyright 2003 by pseudonym67
Everything else Copyright © CodeProject, 1999-2014
Terms of Use
Layout: fixed | fluid