Useful links
1. Introduction
Science and engineering software can have very complicated logics. Unit testing of this software cannot test every execution path of program. This article is devoted to simple paradigm of scientific software testing. This paradigm requires saving calculation result. Testing procedure compares saved data with (new) results of calculation. If modification is correct both results should coincide. This article contains elegant code implementation of this paradigm.
2. Unit testing
Unit testing provides rapid fixing of simple bugs. I think that good testing should include both
- Unit testing
- Complicated testing
This section describes examples of unit testing of formula editor. Any advanced (not only) scientific software operates with formulae. Described here framework contains formula editor. Samples of formulae are presented below. Following picture contains four arithmetic operations:
These and other elementary formulae can be easy tested. Following snippet contains test of these formulae.
[TestMethod]
public void TestMethodBinaryDouble()
{
double type = 0;
Dictionary<string, object> d = new Dictionary<string, object>()
{
{"x", type},
{"y", type}
};
FormulaEditor.VariableDetectors.ExtendedDictionaryVariableDetector det =
new FormulaEditor.VariableDetectors.ExtendedDictionaryVariableDetector(d);
FormulaEditor.Interfaces.IFormulaObjectCreator cr = ExtendedFormulaCreator.GetCreator(det);
XmlDocument doc = new XmlDocument();
doc.LoadXml(Properties.Resources.BinaryDouble);
List<ObjectFormulaTree> l = new List<ObjectFormulaTree>();
XmlNodeList nl = doc.DocumentElement.ChildNodes;
foreach (XmlElement e in nl)
{
MathFormula f = MathFormula.FromString(sizes, e.OuterXml);
ObjectFormulaTree tree = ObjectFormulaTree.CreateTree(f.FullTransform(null), cr);
l.Add(tree);
}
Dictionary<Func<double, double, double>, object[]> dic = new Dictionary<Func<double, double, double>, object[]>()
{
{(double x, double y) => { return x + y;} , new object[]{new object[] {"plus", l[0]}}},
{(double x, double y) => { return x - y;} , new object[]{new object[] {"minus", l[1]}}},
{(double x, double y) => { return x * y;} , new object[]{new object[] {"mult1", l[2]}, new object[] {"mult2", l[3]}}},
{(double x, double y) => { return x / y;} , new object[]{new object[] {"frac", l[4]}}},
{Math.Pow , new object[]{new object[] {"pow", l[5]}}},
{Math.Atan2 , new object[]{new object[] {"atan2", l[6]}}},
};
FormulaEditor.Interfaces.ITreeCollectionProxy proxy = l.ToArray().CreateProxy();
foreach (Func<double, double, double> func in dic.Keys)
{
object[] o = dic[func];
foreach (object[] ob in o)
{
ObjectFormulaTree tree = ob[1] as ObjectFormulaTree;
string fs = ob[0] + "";
GetValue g = proxy[tree];
for (int i = 0; i < 10; i++)
{
double x = 0.007 + 0.07 * i;
det["x"] = x;
for (int j = 0; j < 10; j++)
{
double y = 0.34 + 0.031 * j;
det["y"] = y;
double a = func(x, y);
object b = tree.Result;
Assert.AreEqual(a, b);
proxy.Update();
object c = g();
Assert.AreEqual(a, c);
}
}
}
}
}
This test is very clear. It compares calculatinos obtained from formulae end C# functions.
3. Complicated testing
Unfortunately unit testing is not panacea. Unit testing cannot embrace all complicated tasks. For example
determination of orbits of artificial
Earth satellites contains lots of elements which are presented below:
Main idea of testing such use cases is recording of results. Testing contains comparation of recorded and calculated results. If calculation framework is scalable then testing should have abstract level. Following code contains this absract level.
public interface ITest
{
object this[IComponentCollection collection]
{
get;
}
}
This interface contains one method which tests collection of components and returns tests results.
3.1 Elementary tests
3.1.1 Test of time dependent function
A lot of engineering tasks contains calculation of time dependent functions. Following picture presents calculation of time dependent function
Input signal is transformed by transfer function. Transformation result is time dependent function which is indicated by red curve on Chart
component.
Properties of Input are presented below:
Above picture contains formula of input signal with
Dirac δ function. The Transfer function object has following properties:
It contains formula of transfer function.
Result of transformation is presented below:
Output function has jump at time= 0.5. This jump is caused by Dirac δ function.
Testing of calculation of time dependent function can be performed by following way:
- Saving of time dependent functions;
- Comparing of new calculation with saved result.
Following snippet contains implementation of this test.
[Serializable()]
class LocalChart : ITest, ISerializable
{
#region Fields
Dictionary<string, LocalSeries> series = new Dictionary<string, LocalSeries>();
string name;
string argument;
double start;
double step;
int stepCount;
string[] values = null;
#endregion
#region Ctor
internal LocalChart(string name, double start, double step, int stepCount, string argument, string[] values)
{
this.name = name;
this.start = start;
this.step = step;
this.stepCount = stepCount;
this.argument = argument;
this.values = values;
}
private LocalChart(SerializationInfo info, StreamingContext context)
{
name = info.GetString("Name");
series = info.GetValue("Series", typeof(Dictionary<string, LocalSeries>)) as Dictionary<string, LocalSeries>;
argument = info.GetString("Argument");
start = info.GetDouble("Start");
step = info.GetDouble("Step");
stepCount = info.GetInt32("StepCount");
}
#endregion
#region ISerializable Members
void ISerializable.GetObjectData(SerializationInfo info, StreamingContext context)
{
info.AddValue("Name", name);
info.AddValue("Series", series, typeof(Dictionary<string, LocalSeries>));
info.AddValue("Argument", argument);
info.AddValue("Start", start);
info.AddValue("Step", step);
info.AddValue("StepCount", stepCount);
}
#endregion
#region ITest Members
object ITest.this[IComponentCollection collection]
{
get
{
IDesktop desktop = collection as IDesktop;
IDataConsumer dataconsumer = desktop[name] as IDataConsumer;
Dictionary<string, DataPerformer.Basic.Series> d = GetSeries(collection);
List<string> l = new List<string>();
foreach (string s in d.Keys)
{
if (!series[s].Compare(d[s]))
{
l.Add("Different series values. Object - " + name + ". Series - " + s + ".");
}
}
if (l.Count == 0)
{
return null;
}
return l;
}
}
#endregion
#region Members
internal void Create(IComponentCollection collection)
{
Dictionary<string, DataPerformer.Basic.Series> d = GetSeries(collection);
series.Clear();
foreach (string key in d.Keys)
{
series[key] = new LocalSeries(d[key]);
}
}
Dictionary<string, DataPerformer.Basic.Series> GetSeries(IComponentCollection collection)
{
IDesktop desktop = collection as IDesktop;
IDataConsumer dataConsumer = desktop.GetObject(name) as IDataConsumer;
string[] ss = (values == null) ? series.Keys.ToArray() : values;
return dataConsumer.GetSeries(start, step, stepCount,
argument, ss);
}
internal string Name
{
get
{
return name;
}
}
#endregion
}
Essential part of this code is presented below
object ITest.this[IComponentCollection collection]
{
get
{
IDesktop desktop = collection as IDesktop;
IDataConsumer dataconsumer = desktop[name] as IDataConsumer;
Dictionary<string, DataPerformer.Basic.Series> d = GetSeries(collection);
List<string> l = new List<string>();
foreach (string s in d.Keys)
{
if (!series[s].Compare(d[s]))
{
l.Add("Different series values. Object - " + name + ". Series - " + s + ".");
}
}
if (l.Count == 0)
{
return null;
}
return l;
}
}
Our application can be started with test support. If application is started
with -tc parameter of command line (Aviation.Wpf3D.Advanced.exe -tc) then it supports tests i.e. any saving initialize request of test parameters. User interface of test request is presented below:
This picture means that current scenario is saved with test of Chart component. Saving of this scenario also saves LocalChart object
Loading of scenario also loads and performs test of Chart object.
3.1.2 Test of test
Above chapter contains tests. But we should test work test whether test works. A
test without bug is not test. If we have no but we should make it. Following
snippet contains artificial bug.
AddXY(Converter.ToDouble(x()()), Converter.ToDouble(y()()) + 0.0001);
Adding of 0.0001 is necessary calculation bug. If we load above situation with this bug then we obtain following test report:
3.1.3 Test of nonlinear regression
Nonlinear regression in statistics is the problem of fitting a model.
to multidimensional x, y data, where f is a nonlinear function of x, with regression parameter θ. Vector ε=(ε1,..., εn) is called vector of residuals which.
Regression algorithm defines such vector θ residual parameter σ2=(ε12+...+εn2)/n is minimal. Regression algorithm iteratively estimates new value of θ. During one iteration algorithm defines new value of θ and new value of σ2.
Test of nonlinear regression implies recording number of iterations and residual parameter. Following code snippet contains implementation of this test
[Serializable()]
class RegressionTest : ITest, ISerializable
{
#region Fields
string name;
int number;
double value;
#endregion
#region Ctor
internal RegressionTest(string name, int number)
{
this.name = name;
this.number = number;
}
private RegressionTest(SerializationInfo info, StreamingContext context)
{
name = info.GetString("Name");
number = info.GetInt32("Number");
value = info.GetDouble("Value");
}
#endregion
#region ITest Members
object ITest.this[IComponentCollection collection]
{
get
{
if (GetValue(collection) != value)
{
return "Different regression values. Object - " + name;
}
return null;
}
}
#endregion
#region ISerializable Members
void ISerializable.GetObjectData(SerializationInfo info, StreamingContext context)
{
info.AddValue("Name", name);
info.AddValue("Number", number);
info.AddValue("Value",value);
}
#endregion
#region Members
internal int Number
{
get
{
return number;
}
}
internal string Name
{
get
{
return name;
}
}
internal void Create(IComponentCollection collection)
{
value = GetValue(collection);
}
double GetValue(IComponentCollection collection)
{
IDesktop desktop = collection as IDesktop;
AliasRegression reg = desktop.GetObject(name) as AliasRegression;
for (int i = 0; i < number; i++)
{
reg.FullIterate();
}
return reg.SquareResidual;
}
#endregion
}
This class is serializable. This test object can be saved to stream. Following snippet contains essential part of above implementation
object ITest.this[IComponentCollection collection]
{
get
{
if (GetValue(collection) != value)
{
return "Different regression values. Object - " + name;
}
return null;
}
}
double GetValue(IComponentCollection collection)
{
IDesktop desktop = collection as IDesktop;
AliasRegression reg = desktop.GetObject(name) as AliasRegression;
for (int i = 0; i < number; i++)
{
reg.FullIterate();
}
return reg.SquareResidual;
}
In fact this test compares saved value of σ2 with calculated one.
3.1.4 Example of regression test. Determination of Orbits of Artificial Satellites
Determination of orbits of artificial satellites is considered in my
previous article. Following picture contains elements of orbit determination
Main component is Prosessor which performs determination of orbit.
If we try to save this scenario then obtain following request of test.
This picture has following meaning
- The Processor component is tested
- Count of iteration is 3
If we set to Order default value (=0) then test of processor will be dropped. Saving of this scenario also saves RegressionTest object which contains number of iterations and corresponding σ2. If we load saved file then Processor should perform 3 iterations and compares saved σ2 with calculated one.
Testing of this test can be performed by following artificial bug.
th.UpdateChildrenData();
In case of this artificial following test report will occur.
This sample requires installation of helper files. Following archive should be unpacked to directory of Aviation.Wpf3D.Advanced.exe file.
3.2 Integrated tests
Complicated engineering problems as rule contains a set of elements which need tests. In general test result depends on order of testing. For example time dependency can depend on estimated by regression parameters. In this case result of time dependency test depends on whether estimation was performed before test or after it.
In this chapter some examples of integrated tests is considered.
3.2.1 Chart reconstruction
Let us consider problem of chart reconstruction from image. Sample of image is presented below:
We would like to find math function which corresponds to this image. We also suppose that image contains errors of measurements. Following picture contains elements of this task.
First of all we need scale this image. Moreover X axis and Y axis are not always horizontal and vertical respectively. So one need image scaling in wide sense (scaling with rotation). The image is marked by colored points for scaling. Zero point is scaled by red color. Top and right image points are marked by green and blue color respectively. Detection of these points is performed by following steps. First of all image is filtered.
Following component represents filter or red color
In this formula r, g, b are weights of red, green and blue color respectively.
Colors are normed by following way;
Above formula makes black point on place of red point. Other pixels are white.
Filtration result is contained in Zero Image component. This component is connected to Zero Selection component which transforms image to chart. Properties of this chart are presented below:
This selection enables us to detect coordinates of red point. Nonlinear regression component Zero Regression is used for this purpose (Zero Regression is connected to Zero Selection). Similraly using filters and regression components one can define coordinates of top and left point. Top Processor and Right Processor are being used for this purpose. Usage of filter which defines black pixels enable us define following chart.
After these actions one can define math dependency of chart.
Regression formula is presented below:
Parameters a, b, c, d, k should be estimated. Main Processor is nonlinear regression component which performs estimation.
Following chart contains calculated regression result (red curve) and a set of
approximated points (blue color).
Testing of this scenario should be ordered by following way:
The Order column means order of test:
3.2.2 Test of sounds
Any virtual reality software should support sounds. Also it should support testing of sounds. Let us consider a sample of sound testing.
This sample contains following tasks:
- Determination of orbits of artificial satellites;
- Calculation of motion parameters;
- Blowing of words "up" and "down" at virtual intersection of equator. "Up" is phonated if satellite is moved from South hemisphere to North one and "down" is phonated otherwise.
Following picture represents components of this sample
Left part of above picture contains determination of orbit, right one is related to sounds. Following picture represents right part in details.
This picture has following meaning
| Condition parameter | Meaning | Sound file |
| Sound formula.Formula_1 = true | Satellite intersects equator plane from South to North | up.wav |
| Sound formula.Formula_2 = true | Satellite intersects equator plane from North to South | down.wav |
Testing of sounds is performed by following way. Preparation of test imply saving of dictionary of sounds. Dictionary keys are points of sound times, dictionary values are names of sound files. Implementation of this test is presented below:
[Serializable()]
class SoundTest : ISerializable, ITest
{
#region Fields
string name;
double start;
double step;
int stepCount;
Dictionary<double, string> soundResults = new Dictionary<double, string>();
#endregion
#region Ctor
internal SoundTest(IComponentCollection collection,
string name, double start, double step, int stepCount)
{
this.name = name;
this.start = start;
this.step = step;
this.stepCount = stepCount;
soundResults = GetSounds(collection);
}
private SoundTest(SerializationInfo info, StreamingContext context)
{
name = info.GetString("Name");
start = info.GetDouble("Start");
step = info.GetDouble("Step");
stepCount = info.GetInt32("StepCount");
soundResults = info.GetValue("Sounds", typeof(Dictionary<double, string>)) as Dictionary<double, string>;
}
#endregion
#region ISerializable Members
void ISerializable.GetObjectData(SerializationInfo info, StreamingContext context)
{
info.AddValue("Name", name);
info.AddValue("Start", start);
info.AddValue("Step", step);
info.AddValue("StepCount", stepCount);
info.AddValue("Sounds", soundResults, typeof(Dictionary<double, string>));
}
#endregion
#region ITest Members
object ITest.this[IComponentCollection collection]
{
get
{
List<string> l = new List<string>();
Dictionary<double, string> d = GetSounds(collection);
if (d.Count != soundResults.Count)
{
l.Add("Different number of sounds");
}
foreach (double key in d.Keys)
{
if (!soundResults.ContainsKey(key))
{
l.Add("Illegal time: " + key);
}
else
{
string s = d[key];
if (!s.Equals(soundResults[key]))
{
l.Add("Illegal sound '" + s + "' Time = " + key);
}
}
}
if (l.Count == 0)
{
return null;
}
return l;
}
}
#endregion
#region Members
internal string Name
{
get
{
return name;
}
}
Dictionary<double, string> GetSounds(IComponentCollection collection)
{
IDesktop desktop = collection as IDesktop;
Dictionary<double, string> d = new Dictionary<double, string>();
SoundCollection coll = desktop.GetObject(name) as SoundCollection;
IDataConsumer dc = coll;
ITimeMeasureProvider p = DataPerformer.StaticExtension.Factory.TimeProvider;
Action<string> act = (string s) => { d[p.Time] = s; };
coll.PlaySound += act;
dc.PerformFixed(start, step, stepCount, p,
DifferentialEquationProcessor.Processor, desktop, 0, () => { });
coll.PlaySound -= act;
return d;
}
#endregion
}
If application is started with test support then test component have additional tab page of test's parameters.
This page contains start time, count of steps and step of sound test. Test result depends on operation order. Dictionary of sounds depends on determination of orbit. If we would like save above scenario then we obtain following request of test.
This request has following meaning
- First of all test performs orbit determination with 3 iterations;
- Secondly test of sounds is performed.
This sample requires installation of helper files. Following archive should be unpacked to directory of Aviation.Wpf3D.Advanced.exe file.
4. Points of interests
I find that testing is like to mammillae of man. Testing is not useful and it do not have beauty. However without advanced software is impossible without testing . Testing is inevitable misfortune.
5. Acknowlegment
I would like to acknowledge Weifen Luo for his very useful software. His software is used in my projects.
