Neural Dot Net Pt 4 Neural Net Tester

• Download source files - 1.74 MB
• Visual Studio .NET Files (Not 2003) - 788 Kb

Introduction

The neural net tester is the application that comes with the neural net library and serves to demonstrate that all the networks implemented in the library do exactly what I say; they will do in a provable and repeatable manner. This article is a general view of the testing applications as there are a few ideas floating about for additions, and at the time of writing, there is no guarantee that components won't be rewritten, completely removed or added. So, it is more efficient to have a separate piece dealing with the testing suite, so I don't have to rewrite every article whenever I make a change.

Also, each network will independently describe how to run the tests for the specified network. The reason for this is that it keeps all the information as local as possible, as mistakes will be made if people start getting confused about where the information is about how to test the program.

The Testing application utilizes the Internet Explorer control for displaying HTML pages and is used by the LogViewer component. All my packages use the Sharp Utils DLL, which in this case, primarily contains the classes used for logging.

The Neural Network Tester Program

This is the main testing program for all of the neural network tests that are provided with this package. These tests vary from straight C# implementations of the C++ code in Joey Rogers' book "Object Orientated Neural Networks In C++" to experimental work which I mostly made up as I went along to the implementation of algorithms as described in Simon Hayek's "Neural Networks A Comprehensive Foundation".

The main program allows the running of the test networks and all tests are run on separate threads so that the main application can retain some responsiveness while a neural network test is running. The reasoning behind this is that sometimes the running of the networks can take hours and at times if they are training, they may never satisfactorily complete their training code. There are various reasons for that this can happen and all are noted in the parts that deal with the relevant networks. The threading code is quite simple:

threadStart = new ThreadStart( LoadAndRunAdalineWord );
thread = new Thread( threadStart );

thread.Start();

The ThreadStart and Thread objects are members of the Form class, and they work by passing the name of the thread function to the ThreadStart object and then passing the ThreadStart object to the Thread object. Once Start is called on the Thread, the function that was passed into the ThreadStart object will start. All of the tests in this program use this mechanism.

The Network Output Screen shown above is a simple rich edit box that displays any messages output by the program, with each program being written so that it gives a continuous run down of its current progress. The picture above shows the start of the output for the training of the Adaline one network.

The Menus

• File

  This is the standard idea of a file menu in that it allows you to exit the program or to stop the currently executing thread.

• Generate

  The Generate menu allows you to generate the training files for each individual program where that option is available.

• Run

  The Run menu allows the running of the networks using previously saved networks that are distributed with the release.

• Train

  The Train menu allows you to see how each different network is trained. Some of these can take a long time, and in this section, I have tried to make them train within a reasonable time. This in some cases has meant sacrificing accuracy for training speed and is not a method recommended for networks that are likely to be used in production, but then the idea is that you would train the network before putting it into production and load the previously trained network for the production code.

• Options

  The Options menu is simply there for research by altering things such as the learning rates and the tolerance for errors (BackPropagation Network). It is possible to see how to improve the training times and the accuracy of the training.

The Log Viewer Component

As with the files for the networks, all logging is written to an XML file. This is usually Neural Network Tester.xml for networks that are being run using the Train menu and to a file called Load and Run ... for files generated by programs run from the "Run" menu. The XML log file to be viewed is then selected via the Browse menu and the code then parses the file and writes out HTML files for each class that has written errors to the file before displaying them as tables in separate tab pages.

If the progress option is turned on, there is going to be sometimes ridiculous amounts of data written to the HTML file in a very short time (The standard size for a "Load and run BackPropagation 2" is over a gig) and it can take the code some time to process the generated XML file and turn it into the HTML pages. So, unless you are developing, then it's a good idea to only have this set to warnings and errors. The option for the logging code is set in the main form constructor.

debugLevel = new DebugLevel( DebugLevelSet.WarningsAndErrors );

The DebugLevel can be tested at any point during the code by calling the TestDebugLevel function like so:

if( debugLevel.TestDebugLevel( DebugLevelSet.Progress ) == true )
{
    log.Log( DebugLevelSet.Progress, 
      "Returning original bias of " + 
      NodeValues[ 0 ].ToString(), ClassName );
}

Once the debugging level has been set then any other time that a class wishes to use the debugging facility, it sets up the DebugLevel like so:

debugLevel = new DebugLevel( DebugLevel.currentLevel );

Using the DebugLevel's current level to set the debug level for the class, it is possible to set the debug level so that classes have different debug levels. Although I rarely do this in practice, it is nice to know it's there in case I need it. The LogViewer can also get information from the system event logs if an Event Log Name is specified in the box provided. By Event Log name, I do not mean to suggest any of the Windows system event logs as the Logging class within the SharpUtils DLL has the ability to write to custom event logs or to an XML file or both.

Design Decisions

The reason that the LogViewer is implemented with the list of files on the right hand side is due to the way that C# and the editor implement splitter bars. The original idea was to place the file list in the standard position on the left but what happened was that the way to get the splitter to work is to declare the item on the left first in the control panel and then the item on the left must be docked with a fill style while the item on the right has to be docked to the right with the splitter then added and docked to the right. The only problem with this is that the item on the right is then fairly static in that if you expand the application to full screen, the left hand side of the splitter expands and the right hand size stays the same size. This meant that the file list was then going about two thirds of the way across the screen and the view that contained the HTML page was taking up a third of the screen. This was far too irritating to be allowed to continue, so I switched the two the other way round.

The obvious question here is that surely you should be able to make the view on the right dock as fill and then set the one on the left as docked to the left. You would think that this would work, but every time I've tried it, the splitter control just docks itself to the left edge of the parent and is completely useless when the application is run. I assume that in later versions of Developer Studio .NET, this situation will be improved with either a user interface that does what you want or at the very least, someone giving a clear explanation of what the rules are. But until that happens, we'll just have to work around it.

As with the main part of the program, the LogViewer uses a thread to do its processing. This is done so that application flexibility is maintained and the whole system doesn't freeze while the LogViewer goes through its main loop which you can appreciate, with processing files that have the capability of being over a gigabyte can take some time. The program keeps an eye on the thread through the use of a timer that checks to see if the thread is still active. There are other ways of doing this but all the ones I tried froze the system i.e.:

Using a loop like:

While( thread.IsAlive == true );

has exactly the same effect as not using a thread. There are other ways of checking the thread by checking the thread state but this had the same freezing effect when placed in a loop. So, I use a timer that checks if the thread is alive every so often, and if the thread is finished, it then goes on to build the list of available HTML files on the right hand side bar of the program.

The design for the LogViewer originally allowed it to be more generic than it is turning out to be. Originally, the LogViewer could check the registry to read the event logs and place the contents of the event logs as HTML files along with the files generated from the XML. This facility has largely been marginalized by the fact that the Neural Network Tester doesn't use the registry, maybe when I rewrite it for the fourth time, I'll put it back in. So, as always with these things, it is becoming more application specific than I originally would have wanted.

One final note should be that the LogViewer indiscriminately deletes all HTML files from the working directory before it creates the new ones, so you should avoid placing any HTML document in this folder.

History

• 2 July 2003 :- Initial release
• 27 October 2003 :- Review and edit for CP conformance

References

• Tom Archer (2001) Inside C#, Microsoft Press
• Jeffery Richter (2002) Applied Microsoft .NET Framework Programming, Microsoft Press
• Charles Peltzold (2002) Programming Microsoft Windows With C#, Microsoft Press
• Robinson et al (2001) Professional C#, Wrox
• William R. Staneck (1997) Web Publishing Unleashed Professional Reference Edition, Sams.net
• Robert Callan, The Essence Of Neural Networks (1999) Prentice Hall
• Timothy Masters, Practical Neural Network Recipes In C++ (1993) Morgan Kaufmann (Academic Press)
• Melanie Mitchell, An Introduction To Genetic Algorithms (1999), MIT Press
• Joey Rogers, Object-Orientated Neural Networks in C++ (1997), Academic Press
• Simon Haykin Neural Networks A Comprehensive Foundation (1999), Prentice Hall
• Bernd Oestereich (2002) Developing Software With UML Object-Orientated Analysis And Design In Practice, Addison Wesley
• R Beale & T Jackson (1990) Neural Computing An Introduction, Institute Of Physics Publishing

Thanks

Special thanks go to anyone involved in TortoiseCVS for version control.

All UML diagrams were generated using Metamill version 2.2.