13,201,050 members (54,754 online)
alternative version

#### Stats

14.1K views
15 bookmarked
Posted 1 Jan 2012

# Simple paradigm of scientific software testing

, 1 Jan 2012
 Rate this:

## 1. Introduction

Science and engineering software can have very complicated logics. Unit testing of this software cannot test every execution path of program. This article is devoted to simple paradigm of scientific software testing. This paradigm requires saving calculation result. Testing procedure compares saved data with (new) results of calculation. If modification is correct both results should coincide. This article contains elegant code implementation of this paradigm.

## 2. Unit testing

Unit testing provides rapid fixing of simple bugs. I think that good testing should include both

• Unit testing
• Complicated testing

This section describes examples of unit testing of formula editor. Any advanced (not only) scientific software operates with formulae. Described here framework contains formula editor. Samples of formulae are presented below. Following picture contains four arithmetic operations:

These and other elementary formulae can be easy tested. Following snippet contains test of these formulae.

```/// <summary>
/// Test of double binary
/// </summary>
[TestMethod]
public void TestMethodBinaryDouble()
{
double type = 0;
Dictionary<string, object> d = new Dictionary<string, object>()
{
{"x", type},
{"y", type}
};
FormulaEditor.VariableDetectors.ExtendedDictionaryVariableDetector det =
new FormulaEditor.VariableDetectors.ExtendedDictionaryVariableDetector(d);
FormulaEditor.Interfaces.IFormulaObjectCreator cr = ExtendedFormulaCreator.GetCreator(det);
XmlDocument doc = new XmlDocument();

List<ObjectFormulaTree> l = new List<ObjectFormulaTree>();
XmlNodeList nl = doc.DocumentElement.ChildNodes;
foreach (XmlElement e in nl)
{
MathFormula f = MathFormula.FromString(sizes, e.OuterXml);   // Creation of formula objects
ObjectFormulaTree tree = ObjectFormulaTree.CreateTree(f.FullTransform(null), cr);
}
Dictionary<Func<double, double, double>, object[]> dic = new Dictionary<Func<double, double, double>, object[]>()   // Dictionary of tested
{                                                         // binary functions
{(double x, double y) => { return x + y;} , new object[]{new object[] {"plus",  l[0]}}},
{(double x, double y) => { return x - y;} , new object[]{new object[] {"minus",  l[1]}}},
{(double x, double y) => { return x * y;} , new object[]{new object[] {"mult1",  l[2]}, new object[] {"mult2",  l[3]}}},
{(double x, double y) => { return x / y;} , new object[]{new object[] {"frac",  l[4]}}},
{Math.Pow , new object[]{new object[] {"pow",  l[5]}}},
{Math.Atan2 , new object[]{new object[] {"atan2",  l[6]}}},
};
FormulaEditor.Interfaces.ITreeCollectionProxy proxy = l.ToArray().CreateProxy();   // Creation of proxy code
foreach (Func<double, double, double> func in dic.Keys)
{
object[] o = dic[func];
foreach (object[] ob in o)
{
ObjectFormulaTree tree = ob[1] as ObjectFormulaTree;
string fs = ob[0] + "";
GetValue g = proxy[tree];                                       // Proxy functions
for (int i = 0; i < 10; i++)
{
double x = 0.007 + 0.07 * i;
det["x"] = x;
for (int j = 0; j < 10; j++)
{
double y = 0.34 + 0.031 * j;
det["y"] = y;
double a = func(x, y);                                      // Calculation of functions
object b = tree.Result;                                     // Calculation of tree
Assert.AreEqual(a, b);                                      // Comparation of results
proxy.Update();
object c = g();                                             // Calculation of proxy code
Assert.AreEqual(a, c);                                      // Comparation of results
}
}
}
}
}
```

This test is very clear. It compares calculatinos obtained from formulae end C# functions.

## 3. Complicated testing

Unfortunately unit testing is not panacea. Unit testing cannot embrace all complicated tasks. For example determination of orbits of artificial Earth satellites contains lots of elements which are presented below:

Main idea of testing such use cases is recording of results. Testing contains comparation of recorded and calculated results. If calculation framework is scalable then testing should have abstract level. Following code contains this absract level.

```/// <summary>
/// Test
/// </summary>
public interface ITest
{
/// <summary>
/// Tests collection of components
/// </summary>
/// <param name="collection">Collection of components</param>
/// <returns>Test result</returns>
object this[IComponentCollection collection]
{
get;
}
}
```

This interface contains one method which tests collection of components and returns tests results.

### 3.1 Elementary tests

#### 3.1.1 Test of time dependent function

A lot of engineering tasks contains calculation of time dependent functions. Following picture presents calculation of time dependent function

Input signal is transformed by transfer function. Transformation result is time dependent function which is indicated by red curve on Chart component. Properties of Input are presented below:

Above picture contains formula of input signal with Dirac δ function. The Transfer function object has following properties:

It contains formula of transfer function.

Result of transformation is presented below:

Output function has jump at time= 0.5. This jump is caused by Dirac δ function.

Testing of calculation of time dependent function can be performed by following way:

• Saving of time dependent functions;
• Comparing of new calculation with saved result.

Following snippet contains implementation of this test.

```  /// <summary>
/// Test of time dependent function
/// </summary>
[Serializable()]
class LocalChart : ITest, ISerializable
{

#region Fields

/// <summary>
/// Saved series
/// </summary>
Dictionary<string, LocalSeries> series = new Dictionary<string, LocalSeries>();

string name;

string argument;

double start;

double step;

int stepCount;

string[] values = null;

#endregion

#region Ctor

internal LocalChart(string name, double start, double step, int stepCount, string argument, string[] values)
{
this.name = name;
this.start = start;
this.step = step;
this.stepCount = stepCount;
this.argument = argument;
this.values = values;
}

/// <summary>
/// Loads saved time dependent functions
/// </summary>
/// <param name="info">Serialization info</param>
/// <param name="context">Streaming context</param>
private LocalChart(SerializationInfo info, StreamingContext context)
{
name = info.GetString("Name");
series = info.GetValue("Series", typeof(Dictionary<string, LocalSeries>)) as Dictionary<string, LocalSeries>;
argument = info.GetString("Argument");
start = info.GetDouble("Start");
step = info.GetDouble("Step");
stepCount = info.GetInt32("StepCount");
}

#endregion

#region ISerializable Members

/// <summary>
/// Saves time dependent functions
/// </summary>
/// <param name="info">Serialization info</param>
/// <param name="context">Streaming context</param>
void ISerializable.GetObjectData(SerializationInfo info, StreamingContext context)
{
}

#endregion

#region ITest Members

/// <summary>
/// Tests collection of components
/// </summary>
/// <param name="collection">Collection of components</param>
/// <returns>Test result</returns>
object ITest.this[IComponentCollection collection]
{
get
{
IDesktop desktop = collection as IDesktop;
IDataConsumer dataconsumer = desktop[name] as IDataConsumer;
Dictionary<string, DataPerformer.Basic.Series> d = GetSeries(collection); // Calculation of time dependent function
List<string> l = new List<string>();
foreach (string s in d.Keys)
{
if (!series[s].Compare(d[s])) // Comparation of test results
{
l.Add("Different series values. Object - " + name + ". Series - " + s + ".");
}
}
if (l.Count == 0)
{
return null;
}
return l;
}
}

#endregion

#region Members

/// <summary>
/// Crates tests
/// </summary>
/// <param name="collection">Collection of components</param>
internal void Create(IComponentCollection collection)
{
Dictionary<string, DataPerformer.Basic.Series> d = GetSeries(collection);
series.Clear();
foreach (string key in d.Keys)
{
series[key] = new LocalSeries(d[key]);
}
}

Dictionary<string, DataPerformer.Basic.Series> GetSeries(IComponentCollection collection)
{
IDesktop desktop = collection as IDesktop;
IDataConsumer dataConsumer = desktop.GetObject(name) as IDataConsumer;
string[] ss = (values == null) ? series.Keys.ToArray() : values;
return dataConsumer.GetSeries(start, step, stepCount,
argument, ss);
}

internal string Name
{
get
{
return name;
}
}

#endregion
}
```

Essential part of this code is presented below

``` /// <summary>
/// Tests collection of components
/// </summary>
/// <param name="collection">Collection of components</param>
/// <returns>Test result</returns>
object ITest.this[IComponentCollection collection]
{
get
{
IDesktop desktop = collection as IDesktop;
IDataConsumer dataconsumer = desktop[name] as IDataConsumer;
Dictionary<string, DataPerformer.Basic.Series> d = GetSeries(collection); // Calculation of time dependent function
List<string> l = new List<string>();
foreach (string s in d.Keys)
{
if (!series[s].Compare(d[s])) // Comparation of test results
{
l.Add("Different series values. Object - " + name + ". Series - " + s + ".");
}
}
if (l.Count == 0)
{
return null;
}
return l;
}
}
```

Our application can be started with test support. If application is started with -tc parameter of command line (Aviation.Wpf3D.Advanced.exe -tc) then it supports tests i.e. any saving initialize request of test parameters. User interface of test request is presented below:

This picture means that current scenario is saved with test of Chart component. Saving of this scenario also saves LocalChart object

#### 3.1.2 Test of test

Above chapter contains tests. But we should test work test whether test works. A test without bug is not test. If we have no but we should make it. Following snippet contains artificial bug.

```     //AddXY(Converter.ToDouble(x()()), Converter.ToDouble(y()()));

/*!!! Test of test   (Artificial bug) */

//End test of test*/
```

Adding of 0.0001 is necessary calculation bug. If we load above situation with this bug then we obtain following test report:

#### 3.1.3 Test of nonlinear regression

Nonlinear regression in statistics is the problem of fitting a model.

to multidimensional x, y data, where f is a nonlinear function of x, with regression parameter θ. Vector ε=(ε1,..., εn) is called vector of residuals which. Regression algorithm defines such vector θ residual parameter σ2=(ε12+...+εn2)/n is minimal. Regression algorithm iteratively estimates new value of θ. During one iteration algorithm defines new value of θ and new value of σ2. Test of nonlinear regression implies recording number of iterations and residual parameter. Following code snippet contains implementation of this test

```/// <summary>
/// Test of nonlinear regression
/// </summary>
[Serializable()]
class RegressionTest : ITest, ISerializable
{
#region Fields

/// <summary>
/// Name of component on desktop
/// </summary>
string name;

/// <summary>
/// Number of iterations
/// </summary>
int number;

/// <summary>
/// Residual parameter
/// </summary>
double value;

#endregion

#region Ctor

/// <summary>
/// Constructor
/// </summary>
/// <param name="name">Name of component on desktop</param>
/// <param name="number">Number of iterations</param>
internal RegressionTest(string name, int number)
{
this.name = name;
this.number = number;
}

/// <summary>
/// Deserialization constructor
/// </summary>
/// <param name="info">Serialization info</param>
/// <param name="context">Streaming context</param>
private RegressionTest(SerializationInfo info, StreamingContext context)
{
name = info.GetString("Name");
number = info.GetInt32("Number");
value = info.GetDouble("Value");
}

#endregion

#region ITest Members

/// <summary>
/// Tests collection of components
/// </summary>
/// <param name="collection">Collection of components</param>
/// <returns>Test result</returns>
object ITest.this[IComponentCollection collection]
{
get
{
if (GetValue(collection) != value)  // If calculated value of residual parameter is not equal
// is not equal to saved value of residual parameter
{
return "Different regression values. Object - " + name;  // Then method returns error message
}
return null;        // Null means absence of error
}
}

#endregion

#region ISerializable Members

/// <summary>
/// ISerializable interface implementation
/// </summary>
/// <param name="info">Serialization info</param>
/// <param name="context">Streaming context</param>
void ISerializable.GetObjectData(SerializationInfo info, StreamingContext context)
{
}

#endregion

#region Members

internal int Number
{
get
{
return number;
}
}

internal string Name
{
get
{
return name;
}
}

internal void Create(IComponentCollection collection)
{
value = GetValue(collection);
}

/// <summary>
/// Calculates value of residual parameter
/// </summary>
/// <param name="collection">Collection of objects</param>
/// <returns>Residual parameter</returns>
double GetValue(IComponentCollection collection)
{
IDesktop desktop = collection as IDesktop;
AliasRegression reg = desktop.GetObject(name) as AliasRegression; // Regression component
for (int i = 0; i < number; i++)
{
reg.FullIterate();                  // Iteration cylce
}
return reg.SquareResidual;              // returns residual parameter
}

#endregion

}
```

This class is serializable. This test object can be saved to stream. Following snippet contains essential part of above implementation

``` /// <summary>
/// Tests collection of components
/// </summary>
/// <param name="collection">Collection of components</param>
/// <returns>Test result</returns>
object ITest.this[IComponentCollection collection]
{
get
{
if (GetValue(collection) != value)  // If calculated value of residual parameter is not equal
// is not equal to saved value of residual parameter
{
return "Different regression values. Object - " + name;  // Then method returns error message
}
return null;        // Null means absence of error
}
}

/// <summary>
/// Calculates value of residual parameter
/// </summary>
/// <param name="collection">Collection of objects</param>
/// <returns>Residual parameter</returns>
double GetValue(IComponentCollection collection)
{
IDesktop desktop = collection as IDesktop;
AliasRegression reg = desktop.GetObject(name) as AliasRegression; // Regression component
for (int i = 0; i < number; i++)
{
reg.FullIterate();                  // Iteration cylce
}
return reg.SquareResidual;              // returns residual parameter
}
```

In fact this test compares saved value of σ2 with calculated one.

#### 3.1.4 Example of regression test. Determination of Orbits of Artificial Satellites

Determination of orbits of artificial satellites is considered in my previous article. Following picture contains elements of orbit determination

Main component is Prosessor which performs determination of orbit.

If we try to save this scenario then obtain following request of test.

This picture has following meaning

• The Processor component is tested
• Count of iteration is 3

If we set to Order default value (=0) then test of processor will be dropped. Saving of this scenario also saves RegressionTest object which contains number of iterations and corresponding σ2. If we load saved file then Processor should perform 3 iterations and compares saved σ2 with calculated one. Testing of this test can be performed by following artificial bug.

```th.UpdateChildrenData(); ///TEST!!! Comment this string for artificial bug of orbit determination
```

In case of this artificial following test report will occur.

This sample requires installation of helper files. Following archive should be unpacked to directory of Aviation.Wpf3D.Advanced.exe file.

### 3.2 Integrated tests

Complicated engineering problems as rule contains a set of elements which need tests. In general test result depends on order of testing. For example time dependency can depend on estimated by regression parameters. In this case result of time dependency test depends on whether estimation was performed before test or after it. In this chapter some examples of integrated tests is considered.

#### 3.2.1 Chart reconstruction

Let us consider problem of chart reconstruction from image. Sample of image is presented below:

We would like to find math function which corresponds to this image. We also suppose that image contains errors of measurements. Following picture contains elements of this task.

First of all we need scale this image. Moreover X axis and Y axis are not always horizontal and vertical respectively. So one need image scaling in wide sense (scaling with rotation). The image is marked by colored points for scaling. Zero point is scaled by red color. Top and right image points are marked by green and blue color respectively. Detection of these points is performed by following steps. First of all image is filtered. Following component represents filter or red color

In this formula r, g, b are weights of red, green and blue color respectively. Colors are normed by following way;

• |r|≤1;
• |g|≤1;
• |b|≤1.

Above formula makes black point on place of red point. Other pixels are white. Filtration result is contained in Zero Image component. This component is connected to Zero Selection component which transforms image to chart. Properties of this chart are presented below:

This selection enables us to detect coordinates of red point. Nonlinear regression component Zero Regression is used for this purpose (Zero Regression is connected to Zero Selection). Similraly using filters and regression components one can define coordinates of top and left point. Top Processor and Right Processor are being used for this purpose. Usage of filter which defines black pixels enable us define following chart.

After these actions one can define math dependency of chart. Regression formula is presented below:

Parameters a, b, c, d, k should be estimated. Main Processor is nonlinear regression component which performs estimation. Following chart contains calculated regression result (red curve) and a set of approximated points (blue color).

Testing of this scenario should be ordered by following way:

The Order column means order of test:

#### 3.2.2 Test of sounds

Any virtual reality software should support sounds. Also it should support testing of sounds. Let us consider a sample of sound testing. This sample contains following tasks:

• Determination of orbits of artificial satellites;
• Calculation of motion parameters;
• Blowing of words "up" and "down" at virtual intersection of equator. "Up" is phonated if satellite is moved from South hemisphere to North one and "down" is phonated otherwise.

Following picture represents components of this sample

Left part of above picture contains determination of orbit, right one is related to sounds. Following picture represents right part in details. This picture has following meaning

 Condition parameter Meaning Sound file Sound formula.Formula_1 = true Satellite intersects equator plane from South to North up.wav Sound formula.Formula_2 = true Satellite intersects equator plane from North to South down.wav

Testing of sounds is performed by following way. Preparation of test imply saving of dictionary of sounds. Dictionary keys are points of sound times, dictionary values are names of sound files. Implementation of this test is presented below:

``` /// <summary>
/// Test of sound
/// </summary>
[Serializable()]
class SoundTest : ISerializable, ITest
{

#region Fields

string name;

double start;

double step;

int stepCount;

/// <summary>
/// Dictionary of sounds
/// Keys are instants of sounds
/// Values are sound filenames
/// </summary>
Dictionary<double, string> soundResults = new Dictionary<double, string>();

#endregion

#region Ctor

/// <summary>
/// Constructor
/// </summary>
/// <param name="collection">Collection of components</param>
/// <param name="name">Sound component name</param>
/// <param name="start">Start time</param>
/// <param name="step">Step</param>
/// <param name="stepCount">Count of steps</param>
internal SoundTest(IComponentCollection collection,
string name, double start, double step, int stepCount)
{
this.name = name;
this.start = start;
this.step = step;
this.stepCount = stepCount;
soundResults = GetSounds(collection);
}

/// <summary>
/// Deserialization constructor
/// </summary>
/// <param name="info">Serialization info</param>
/// <param name="context">Streaming context</param>
private SoundTest(SerializationInfo info, StreamingContext context)
{
name = info.GetString("Name");
start = info.GetDouble("Start");
step = info.GetDouble("Step");
stepCount = info.GetInt32("StepCount");
soundResults = info.GetValue("Sounds", typeof(Dictionary<double, string>)) as Dictionary<double, string>;
}

#endregion

#region ISerializable Members

/// <summary>
/// ISerializable interface implementation
/// </summary>
/// <param name="info">Serialization info</param>
/// <param name="context">Streaming context</param>
void ISerializable.GetObjectData(SerializationInfo info, StreamingContext context)
{

// Dictionary of sounds saving
}

#endregion

#region ITest Members

/// <summary>
/// Testing
/// </summary>
/// <param name="collection">Collection of objects</param>
/// <returns>Tesing result</returns>
object ITest.this[IComponentCollection collection]
{
get
{
List<string> l = new List<string>();
Dictionary<double, string> d = GetSounds(collection); // Calculates dictionary of sounds

// Comparation of saved dictionary with calculated one
if (d.Count != soundResults.Count)
{
}
foreach (double key in d.Keys)
{
if (!soundResults.ContainsKey(key))
{
}
else
{
string s = d[key];
if (!s.Equals(soundResults[key]))
{
l.Add("Illegal sound '" + s + "' Time = " + key);
}

}
}
if (l.Count == 0)
{
return null;
}
// Returns list of errors
return l;
}
}

#endregion

#region Members

/// <summary>
/// Name
/// </summary>
internal string Name
{
get
{
return name;
}
}

/// <summary>
/// Calculates dictionary of sounds
/// </summary>
/// <param name="collection">Collection of objects</param>
/// <returns>Dictionary of sounds</returns>
Dictionary<double, string> GetSounds(IComponentCollection collection)
{
IDesktop desktop = collection as IDesktop;
Dictionary<double, string> d = new Dictionary<double, string>();
SoundCollection coll = desktop.GetObject(name) as SoundCollection;
IDataConsumer dc = coll;
ITimeMeasureProvider p = DataPerformer.StaticExtension.Factory.TimeProvider;
Action<string> act = (string s) => { d[p.Time] = s; };
coll.PlaySound += act;
dc.PerformFixed(start, step, stepCount, p,
DifferentialEquationProcessor.Processor, desktop, 0, () => { });
coll.PlaySound -= act;
return d;
}

#endregion
}
```

If application is started with test support then test component have additional tab page of test's parameters.

This page contains start time, count of steps and step of sound test. Test result depends on operation order. Dictionary of sounds depends on determination of orbit. If we would like save above scenario then we obtain following request of test.

This request has following meaning

• First of all test performs orbit determination with 3 iterations;
• Secondly test of sounds is performed.

This sample requires installation of helper files. Following archive should be unpacked to directory of Aviation.Wpf3D.Advanced.exe file.

## 4. Points of interests

I find that testing is like to mammillae of man. Testing is not useful and it do not have beauty. However without advanced software is impossible without testing . Testing is inevitable misfortune.

## 5. Acknowlegment

I would like to acknowledge Weifen Luo for his very useful software. His software is used in my projects.

## Share

 Architect Russian Federation
Ph. D. Petr Ivankov worked as scientific researcher at Russian Mission Control Centre since 1978 up to 2000. Now he is engaged by Aviation training simulators http://dinamika-avia.com/ . His additional interests are:

1) Noncommutative geometry

http://front.math.ucdavis.edu/author/P.Ivankov

2) Literary work (Russian only)

http://zhurnal.lib.ru/editors/3/3d_m/

3) Scientific articles
http://arxiv.org/find/all/1/au:+Ivankov_Petr/0/1/0/all/0/1

## You may also be interested in...

 Pro Pro

 First Prev Next
 My vote of 5 Kanasz Robert28-Sep-12 5:49 Kanasz Robert 28-Sep-12 5:49
 My vote of 5 manoj kumar choubey14-Mar-12 22:27 manoj kumar choubey 14-Mar-12 22:27
 My vote of 5 Filip D'haene2-Jan-12 12:47 Filip D'haene 2-Jan-12 12:47
 MALWARE BigTimber@home1-Jan-12 8:35 BigTimber@home 1-Jan-12 8:35
 Re: MALWARE Petr Ivankov1-Jan-12 8:58 Petr Ivankov 1-Jan-12 8:58
 Exploit not fixed agorby3-Jan-12 0:50 agorby 3-Jan-12 0:50