Abstract Nonsense in Software Development. Databases





5.00/5 (1 vote)
Application of abstract approach to database domain
- Download Database demo project source code - 4 MB
- Download additional files - 2.4 KB
- Download a sample database dump - 220 KB
- Download a generator of Hipparcos and Tycho star catalogue - 134 KB
- Download a sample SQL Server database dump - 220 KB
- Download a database regression sample - 31 KB
- Download a sample of iterated regression - 30.5 KB
- Download a sample of 2D charts - 453 KB
- Download a sample of 3D charts - 137 KB
- Download a sample of database + digital image processing - 367 KB
- Download a sample of NASA/IPAC Extragalactic Database - 12.9 KB
Useful links
- Universal engineering software homepage
- Original article
- Abstract Nonsense in Software Development. Outlook
- Abstract Nonsense in Software Development. Real-time
- Microsoft Visual Studio Express
- SQL Server Express
- Oracle Express Edition
- Category theory
- NASA/IPAC Extragalactic Database
- The Hipparcos and Tycho Catalogues
1 Introduction.
This article contains further development of my articles devoted to common problems and real-time. This article depends on previous ones. My samples are accomplished with "serialized" samples where data sets are serialized. Usage of these samples do not require SQL server databases.
IMPORTANT: The demo solution contains the OracleTableProvider
project. This project is intended for support of the Oracle client. If the Oracle client is not installed then the OracleTableProvider
project should be excluded from the solution.
2 Background
The Database domain contains following base types.
IDataSetProvider
this interface is implemented by any object which supplies aDataSet
.IDataSetConsumer
this interface is implemented by any object which usesDataSet
.DataSetArrow
. Source (resp. target) of this arrow should implementIDataSetConsumer
(resp.IDataSetProvider
interface).
Following code contains these base types.
/// <summary> /// Provider of data set /// </summary> public interface IDataSetProvider { /// <summary> /// Provided data set /// </summary> DataSet DataSet { get; } /// <summary> /// Factory. This object can be null. It is not null for databases (SQL Server, Oracle, ...) /// </summary> IDataSetFactory Factory { get; set; } /// <summary> /// Change event /// </summary> event Action<DataSet> Change; }
/// <summary> /// Data set consumer /// </summary> public interface IDataSetConsumer { /// <summary> /// Adds data set /// </summary> /// <param name="dataSet">Data set to add</param> void Add(DataSet dataSet); /// <summary> /// Removes data set /// </summary> /// <param name="dataSet">Data set to remove</param> void Remove(DataSet dataSet); /// <summary> /// Factory /// </summary> IDataSetFactory Factory { get; set; } /// <summary> /// Add event /// </summary> event Action<DataSet> OnAdd; /// <summary> /// Add event /// </summary> event Action<DataSet> OnRemome; }
/// <summary> /// Arrow between data set provider and data set consumer /// </summary> [SerializableAttribute()] public class DataSetArrow : CategoryArrow, ISerializable, IRemovableObject { #region Fields /// <summary> /// Source /// </summary> protected IDataSetConsumer source; /// <summary> /// Target /// </summary> protected IDataSetProvider target; #endregion #region Constructors /// <summary> /// Default constructor /// </summary> public DataSetArrow() { } /// <summary> /// Deserialization constructor /// </summary> /// <param name="info">Serialization info</param> /// <param name="context">Streaming context</param> public DataSetArrow(SerializationInfo info, StreamingContext context) { } #endregion #region ISerializable Members /// <summary> /// ISerializable interface implementation /// </summary> /// <param name="info">Serialization info</param> /// <param name="context">Streaming context</param> public void GetObjectData(SerializationInfo info, StreamingContext context) { } #endregion #region ICategoryArrow Members /// <summary> /// The source of this arrow /// </summary> public override ICategoryObject Source { get { return source as ICategoryObject; } set { source = value.GetSource<IDataSetConsumer>(); } } /// <summary> /// The target of this arrow /// </summary> public override ICategoryObject Target { get { return target as ICategoryObject; } set { target = value.GetTarget<IDataSetProvider>(); source.Factory = target.Factory; source.Add(target.DataSet); } } #endregion #region IRemovableObject Members /// <summary> /// The post remove operation /// </summary> public void RemoveObject() { if (source != null & target != null) { if (target.DataSet != null) { source.Remove(target.DataSet); } } } #endregion }
Following picture shows example of these basic objects.

The IPAC is a provider of a data set obtained from NASA/IPAC Extragalactic Database. The Chart is a consumer of the data set. The Link is an object of the DataSetArrow
type. Separation of data set provides from data set consumers is an implementation of the bridge pattern, following picture shows how different consumers can be connected to different providers.
Any data set provider can be replaced by another one. Following 3 pictures represent the same data set obtained from Sql Server, Oracle and serialized data set respectively.
My samples are accomplished by serialized data sets for autonomy.
3 Provides of data sets
3.1 Data sets from databases
Data sets can be obtained from different databases (SQL Server, Oracle, ...). In this case the Factory
property of IDataSetProvider
should not be null. This property is of IDataSetFactory
type.
/// <summary> /// Factory for creating data set metatata from connection /// </summary> public interface IDataSetFactory { /// <summary> /// Name of factory /// </summary> string FactoryName { get; } /// <summary> /// Creates connection /// </summary> DbConnection Connection { get; } /// <summary> /// Command /// </summary> DbCommand Command { get; } /// <summary> /// Gets metadata data set from connection /// </summary> /// <param name="connection">The connection</param> /// <returns>The metadata data set</returns> DataSet GetData(DbConnection connection); /// <summary> /// Gets metadata data set from connection string /// </summary> /// <param name="connectionString">The connection</param> /// <returns>The metadata data set</returns> DataSet GetData(string connectionString); /// <summary> /// Data adapter /// </summary> IDbDataAdapter Adapter { get; } // Full code is contained in the source file }
Following class diagram shows different implementations of above interface.
Following code represents implementation of this interface for SQL Server and Oracle.
/// <summary> /// Factory of SQL Server /// </summary> public class SQLServerFactory : IDataSetFactory { #region IDataSetFactory Members /// <summary> /// Name of factory /// </summary> string IDataSetFactory.FactoryName { get { return "SQL Server"; } } /// <summary> /// Creates connection /// </summary> System.Data.Common.DbConnection IDataSetFactory.Connection { get { return new SqlConnection(); } } /// <summary> /// Command /// </summary> System.Data.Common.DbCommand IDataSetFactory.Command { get { return new SqlCommand(); } } // Full code is contained in the source file #endregion }
/// <summary> /// Oracle data set factory /// </summary> public class OracleFactory : IDataSetFactory { public static readonly OracleFactory Singleton = new OracleFactory(); private OracleFactory() { } #region IDataSetFactory Members /// <summary> /// Creates connection /// </summary> public System.Data.Common.DbConnection Connection { get { return new OracleConnection(); } } /// <summary> /// Command /// </summary> public System.Data.Common.DbCommand Command { get { return new OracleCommand(); } } // Full code is contained in the source file #endregion }
3.2 Data sets from files
The DataSet class is serializable. The SavedDataProvider
just serialize data set. Following code represents its implementation.
/// <summary> /// Data provider from xml /// </summary> [Serializable()] public class SavedDataProvider : CategoryObject, ISerializable, IDataSetProvider { #region Fields /// <summary> /// Data set /// </summary> protected DataSet dataSet = new DataSet(); /// <summary> /// Change event /// </summary> protected Action<DataSet> change = (DataSet ds) => { }; #endregion #region Ctor /// <summary> /// Default Constructor /// </summary> public SavedDataProvider() { } /// <summary> /// Deserialization constructor /// </summary> /// <param name="info">Serialization info</param> /// <param name="context">Streaming context</param> protected SavedDataProvider(SerializationInfo info, StreamingContext context) { dataSet = info.Deserialize<DataSet>("DataSet"); } #endregion #region ISerializable Members /// <summary> /// ISerializable interface implementation /// </summary> /// <param name="info">Serialization info</param> /// <param name="context">Streaming context</param> public virtual void GetObjectData(SerializationInfo info, StreamingContext context) { info.Serialize<DataSet>("DataSet", dataSet); } #endregion #region IDataSetProvider Members DataSet IDataSetProvider.DataSet { get { return dataSet; } } IDataSetFactory IDataSetProvider.Factory { get { return null; } set { } } event Action<DataSet> IDataSetProvider.Change { add { change += value; } remove { change -= value; } } #endregion #region Members /// <summary> /// Sets Data set /// </summary> /// <param name="dataSet"></param> public void Set(DataSet dataSet) { this.dataSet = dataSet; change(dataSet); } #endregion }
4.3 External data sets
Data sets can be obtained from different sources. For example data set can be obtained from NASA/IPAC Extragalactic Database. I have developed ExternalDataSetProvider
class for this purpose.
/// <summary> /// External data set provider /// </summary> [Serializable()] public class ExternalDataSetProvider : SavedDataProvider, IChildrenObject { #region Fields /// <summary> /// Factory of data set /// </summary> IDataSetPoviderFactory factory; /// <summary> /// Type name of factory /// </summary> string factoryType; /// <summary> /// Url /// </summary> string url = ""; IAssociatedObject[] children = new IAssociatedObject[1]; #endregion #region Ctor /// <summary> /// Constructor from type of child object /// </summary> /// <param name="factoryType">Type of factory</param> public ExternalDataSetProvider(string factoryType) { this.factoryType = factoryType; CreateFactory(); } /// <summary> /// Deserialization constructor /// </summary> /// <param name="info">Serialization info</param> /// <param name="context">Streaming context</param> protected ExternalDataSetProvider(SerializationInfo info, StreamingContext context) : base(info, context) { factoryType = info.GetString("Factory"); url = info.GetString("Url"); CreateFactory(); } #endregion #region ISerializable Members /// <summary> /// ISerializable interface implementation /// </summary> /// <param name="info">Serialization info</param> /// <param name="context">Streaming context</param> public override void GetObjectData(SerializationInfo info, StreamingContext context) { base.GetObjectData(info, context); info.AddValue("Factory", factoryType); info.AddValue("Url", url); } #endregion #region IChildrenObject Members IAssociatedObject[] IChildrenObject.Children { get { return children; } } #endregion #region Private Members /// <summary> /// Creates factory /// </summary> void CreateFactory() { Type t = Type.GetType(factoryType); if (t != null) { // Constructor of child object ConstructorInfo c = t.GetConstructor(new Type[0]); factory = c.Invoke(new object[0]) as IDataSetPoviderFactory; dataSet = factory.GetData(this.url); factory.Change += (string url) => { if (this.url.Equals(url)) { return; } this.url = url; dataSet = factory.GetData(url); change(dataSet); }; children[0] = factory as IAssociatedObject; } } #endregion
Above code requires some comments. This class contains a child object factory
of the IDataSetPoviderFactory
type. The child is serialized together with its parent. It is created by its name of type. Following code represents the IDataSetPoviderFactory
type.
/// <summary> /// Factory of data set provider /// </summary> public interface IDataSetPoviderFactory { /// <summary> /// Name /// </summary> string Name { get; } /// <summary> /// Gets data /// </summary> /// <param name="url">Url</param> /// <returns>Data</returns> DataSet GetData(string url); /// <summary> /// Change event /// </summary> event Action<string> Change; }
Object of this type supplies necessary data set. Following code provides implementation of this interface.
namespace Nasa.Ipac.Extragalactic { /// <summary> /// Data Provider /// </summary> [Diagram.UI.Attributes.Url("http://ned.ipac.caltech.edu/forms/nearposn.html")] public class DataProvider : CategoryObject, IDataSetPoviderFactory, IUrlConsumer, IUrlProvider { #region Fields private event Action<string> changeData = (string url) => { }; private DataSet dataSet = new DataSet(); private string url = ""; private Action<string> changeUrlConsumer = (string url) => { }; private Action<string> changeUrlProvider = (string url) => { }; #endregion #region IDataSetPoviderFactory Members string IDataSetPoviderFactory.Name { get { return "NASA/IPAC Extragalactic Database"; } } DataSet IDataSetPoviderFactory.GetData(string url) { UpdateData(url); return dataSet; } event Action<string> IDataSetPoviderFactory.Change { add { changeData += value; } remove { changeData -= value; } } #endregion #region IUrlConsumer Members string IUrlConsumer.Url { set { UpdateData(value); } } event Action<string> IUrlConsumer.Change { add { changeUrlConsumer += value; } remove { changeUrlConsumer -= value; } } #endregion #region IUrlProvider Members string IUrlProvider.Url { get { return url; } } event Action<string> IUrlProvider.Change { add { changeUrlProvider += value; } remove { changeUrlProvider -= value; } } #endregion #region Private Members void UpdateData(string url) { if (url == null) { return; } if (!url.Equals(this.url)) { DataTable dt = url.GetDataTable(); this.url = url; dataSet = new DataSet(); dataSet.Tables.AddRange(new DataTable[] { dt }); changeData(url); } } #endregion } } namespace Nasa.Ipac.Extragalactic.Data { /// <summary> /// Gets data from NASA Astrogalactic website /// </summary> public static class StaticExtensionNasaIpacExtragalacticData { #region Public Members /// <summary> /// Gets data table from http://ned.ipac.caltech.edu/forms/nearposn.html /// </summary> /// <param name="url">Url</param> /// <returns>Data Table</returns> static public DataTable GetDataTable(this string url) { DataTable table = new DataTable(); lock (ob) { /// Web request WebRequest req = WebRequest.Create(url); WebResponse resp = req.GetResponse(); List<string> l = new List<string>(); // reads data using (TextReader reader = new StreamReader(resp.GetResponseStream())) { return reader.GetDataTable(); } } } /// <summary> /// Gets data table from reader /// </summary> /// <param name="reader">Reader</param> /// <returns>Data table</returns> public static DataTable GetDataTable(this TextReader reader) { // Full code is contained in the source file } #endregion } }
Following pictures explain interoperability with NASA/IPAC Extragalactic Database
4 Usage of Plug-ins
4.1 Business logic
It is clear that that the NASA/IPAC Extragalactic Database is very special component. So it should be implemented as plug-in. All plugins of my software are described in XML file.
<?xml version="1.0" encoding="utf-8" ?> <Root> <Assemblies> <Assembly file="Nasa.Ipac.Extragalactic.Data.dll"/> <Assembly file="Nasa.Ipac.Extragalactic.dll"/> </Assemblies> <Page pageName="Web databases" pageHint="Databaes obtained by Web"> <Object icon="Containers\NED.ico" type="DataSetService.ExternalDataSetProvider,DataSetService, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null" param="Nasa.Ipac.Extragalactic.DataProvider,Nasa.Ipac.Extragalactic, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null" hint="Extragalactic" arrow="false" /> </Page> </Root>
Above XML means that application should load followind additional libraries:
Nasa.Ipac.Extragalactic.Data.dll
Nasa.Ipac.Extragalactic.dll
The Page
tag is responsible for following tab page.
The Object
tag is responsible for a button. The icon
attribute corresponds to button's icon, the type
attribute corresponds to type of the object. The param
attribute corresponds to a kind of the object. Meaning of the kind depends on context. This attribute can be used by Reflection as it is presented below.
if (kind.Length > 0) // Kind or additional parameter { // Searches constructor from string ConstructorInfo ci = t.GetConstructor(new System.Type[] { typeof(string) }); if (ci != null) { // Creates an object ICategoryObject ob = ci.Invoke(new object[] { kind }) as ICategoryObject; return ob; // returns the object } }
The ExternalDataSetProvider
has following constructor from string
/// <summary> /// Constructor from type of child object /// </summary> /// <param name="factoryType">Type of factory</param> public ExternalDataSetProvider(string factoryType) { this.factoryType = factoryType; Type t = Type.GetType(factoryType); if (t != null) { // Constructor of child object ConstructorInfo c = t.GetConstructor(new Type[0]); factory = c.Invoke(new object[0]) as IDataSetPoviderFactory; } }
so following XML code
<Object type="DataSetService.ExternalDataSetProvider,DataSetService, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null" param="Nasa.Ipac.Extragalactic.DataProvider,Nasa.Ipac.Extragalactic, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null" />
implies following C# code
new DataSetService.ExternalDataSetProvider(new Nasa.Ipac.Extragalactic.DataProvider())
However there are other meanings of the kind, for example, as it is presented below.
/// <summary> /// Creates object the corresponds to button /// </summary> /// <param name="button">The button</param> /// <returns>Created object</returns> public override ICategoryObject CreateObject(IPaletteButton button) { string kind = button.Kind; // Kind of the object Type type = button.ReflectionType; if (type.IsSubclassOf(typeof(Camera))) { return factory.NewCamera(); } if (type.Equals(typeof(Motion6D.SerializablePosition))) { object ob = factory.CreateObject(kind); // Usage of the kind if (ob != null) { SerializablePosition pos = new SerializablePosition(); pos.Parameters = ob; if (ob is IPositionObject) { IPositionObject po = ob as IPositionObject; po.Position = pos; } return pos; } } return null; }
4.2 Automatic extension of functionality
If the Oracle client is installed on your computer and you would like use it within my framework. You should just put OracleTableProvider.dll
to the application directory. Exclusion of OracleTableProvider.dll
reduces functionality only. Following function searches all objects which implement the IDataSetFactory
interface
/// <summary> /// Initialization of database drivers /// </summary> /// <param name="path">Path of drivers</param> void Initialize(string path) { // Gets all database drivers from this directory IEnumerable<IDataSetFactory> en = path.GetInterfaces<IDataSetFactory>(); List<string> l = new List<string>(); foreach (IDataSetFactory f in en) { factories[f.FactoryName] = f; // Dictinary of drivers } List<string> ln = new List<string>(factories.Keys); ln.Sort(); names = ln.ToArray(); // Ordered names of drivers }
where the GetInterfaces
is presented below.
/// <summary> /// Gets interfaces /// </summary> /// <typeparam name="T">Interface type</typeparam> /// <param name="directory">Directory</param> /// <returns>Interface objects</returns> public static IEnumerable<T> GetInterfaces<T>(this string directory) where T : class { string[] fn = Directory.GetFiles(directory, "*.dll"); // Dll files Assembly[] ass = AppDomain.CurrentDomain.GetAssemblies(); // Current domain assemblies List<string> l = new List<string>(); foreach (Assembly a in ass) // Looking for objects in loaded assemblies { l.Add(a.Location); IEnumerable<T> en = a.GetInterfaces<T>(); foreach (T t in en) { yield return t; } } foreach (string f in fn) // Looking for objects in directory { if (!l.Contains(f)) { IEnumerable<T> en = Assembly.LoadFile(f).GetInterfaces<T>(); foreach (T t in en) { yield return t; } } } } /// <summary> /// Gets interfaces from assembly (Searching of driver assembly) /// </summary> /// <typeparam name="T">Interface type</typeparam> /// <param name="assembly">Assembly</param> /// <returns>Interface objects</returns> public static IEnumerable<T> GetInterfaces<T>(this Assembly assembly) where T : class { Type[] types = new Type[0]; try { types = assembly.GetTypes(); // Assembly types } catch (Exception) { } string st = typeof(T).FullName; // Interface full name foreach (Type t in types) { Type ti = t.GetInterface(st); // Gets interface if (ti == null) { continue; } FieldInfo fi = t.GetField("Singleton"); // Gets Singleton field if (fi != null) { yield return fi.GetValue(null) as T; // yeild Singleton } } }
4.3 User interface
Although my framework does not know about Nasa.Ipac.Extragalactic.DataProvider
type however my framework supplies edition of properties of this object. User interface contains two tab pages.
The Data table page corresponds to parent object of ExternalDataSetProvider
type. The Data source page corresponds two child object of following class.
namespace Nasa.Ipac.Extragalactic { ////// Data Provider /// [Diagram.UI.Attributes.Url("http://ned.ipac.caltech.edu/forms/nearposn.html")] // Initial url public class DataProvider : CategoryObject, IDataSetPoviderFactory, IUrlConsumer, IUrlProvider { // Implemetation ... }
The [Diagram.UI.Attributes.Url("http://ned.ipac.caltech.edu/forms/nearposn.html")]
is responsible for a Search tab page, the IUrlConsumer
, IUrlProvider
interfaces are responsible for Result tab pages.
First page is responsible for constant URL, second one corresponds to changed URL. Following code represents creation of child user interface.
// Searching of additional control object o = factory.GetAdditionalFeature<IUrlConsumer>(provider as IAssociatedObject); if (o is Control) // Additional control { form = new FormExternalData(this.GetRootLabel(), o as Control); return; }
The o
object is used as child tab page on the FormExternalData
. The GetAdditionalFeature
function if presented below.
/// <summary> /// Gets additional feature /// </summary> /// <typeparam name="T">Feature type</typeparam> /// <param name="factory">User interface factory</param> /// <param name="obj">Obj</param> /// <returns>Feature</returns> static public object GetAdditionalFeature<T>(this IUIFactory factory, IAssociatedObject obj) { IUIFactory f = factory; IUIFactory p = factory.Parent; if (p != null) { f = p; } if (obj == null) { return null; } if (obj is T) { return f.GetAdditionalFeature<T>((T)obj); } if (obj is IChildrenObject) // If object has children { IAssociatedObject[] ao = (obj as IChildrenObject).Children; foreach (IAssociatedObject aa in ao) // Searches additional feature among children { object ob = GetAdditionalFeature<T>(f, aa); if (ob != null) { return ob; } } } return null; }
Following code represents implementation of GetAdditionalFeature
for an implementation of IUIFactory
interface.
/// <summary> /// Gets additional feature /// </summary> /// <typeparam name="T">Feature type</typeparam> /// <param name="obj">Object</param> /// <returns>Feature</returns> public override object GetAdditionalFeature<T>(T obj) { if (!typeof(T).Equals(typeof(IUrlConsumer))) { return null; } IUrlConsumer c = obj as IUrlConsumer; UserControls.UserControlUrl uc = new UserControls.UserControlUrl(); if (c is IUrlProvider) { uc.Set(c as IUrlProvider); } uc.Set(c); return uc; }
Above code means that if child object implements IUrlConsumer
interface then editor form contains a control of the UserControlUrl
type. So user interface is automatically constructed from interfaces.
5. Low cohesion
My soft contains a lot of libraries. However this fact promotes low cohesion and this fact supplies a lot of benefits. For example the Nasa.Ipac.Extragalactic.Data.dll
supplies creation of data sets, but this library is fully independent of my framework. So this library can be easy integrated into other applications. The Nasa.Ipac.Extragalactic.dll
is a bridge between Nasa.Ipac.Extragalactic.Data.dll
and my framework.
There are many levels of cohesion. For example in one of my projects I use Empirical, global model of the Earth's atmosphere from ground to space (NRLMSISE-00) source code. This code is not object oriented. I developed an object oriented wrapper because I would like simultaneously use several copies of atmosphere calculations. Following table represent
5 levels of cohesion.
Level | Name | Dependencies | Functionality | Usage |
1 | Msise C++ class | No | Calculation of atmospheric parameters | Any platform which supports C++ |
2 | MSISEAtmosphere.dll managed C++ project | .NET | .NET wrapper (bridge) of Msise | Any platform which supports .NET |
3 | DynamicAtmosphere.MSISE.dll C# project | .NET, BaseTypes.dll | Wrapper of previous project with support of different physical units | Any platform which supports .NET |
4 | DynamicAtmosphere.MSISE.Wrapper.dll C# project | .NET, my framework | Bridge between my framework an atmospheric calculations | My framework |
5 | DynamicAtmosphere.MSISE.Wrapper.UI.dll C# project | .NET, System.Windows.Forms, my framework | System.Windows.Forms user interface for atmosphere calculation | My framework supplied with a System.Windows.Forms user interface |
You can download this code from my article.
6 Consumers of data sets
6.1 Statistical selections from data sets
In my previous article I have considered statistics. Data sets can be used as statistical data sets. My software uses two schemes of statistical analysis which are explained below.
6.1.1 Loading of the full selection
This scheme imply loading selection at once.
The processor compares Calculated parameters with Selection, calculates residuals, and then corrects Regression parameters. Following example shows an application of data set for a nonlinear regeression.
The Data is object of a StatementWrapper
type which implements the IDataSetProvider
interface and encapsulates SQL queries. Properties of the Data are presented below.
This object uses the SQL Server and performs following query:
SELECT x.x, y.y, z.z, f1.f1, f2.f2 FROM f1, f2, x, y, z WHERE x.Id = f1.Idx AND y.Id = f1.Idy AND z.Id = f1.Idz AND f1.Idx = f2.Idx AND f1.Idy = f2.Idy AND f1.Idz = f2.Idz
In result it provides a data set. The SQL server data set can be replaced with an Oracle one as it is presented above. The DS link associates a IDataSetProvider
with a IDataSetConsumer
. The Selection is object of the DataSetSelection
type. This type also implements both IMeasurements
and IStructuredSelectionCollection
. The IMeasurements
is an interface of the information flow. Outputs of the Selection object correspond to columns of the data set table.
The table contains 1000 rows and all 5 columns are of Double type, so the Selection return five objects of Double[1000]
"type". These objects are used by the Formula.
The Formula object performs component-wise calculation of vector functions. The IStructuredSelectionCollection
is an interface of the statistics domain. The Selection object provides 5 statistical selections which correspond to 5 columns of data table. The Stat arrow is an object of the SelectionLink
type which is described here. The arrow corresponds to the statistics domain. Properties of the Regression object are presented below:
These properties have following meaning. We would like to find values of a, b, c, d, g of the Formula such that Formula_1, Formula_2 approximate selections Table_f1 and Table_f2 respectively. Following picture represents values of parameters before and after approximation.
6.1.2 Iterative scheme
Sometimes previous scheme requires huge random-access memory, because it simultaneously loads all data. The iterative scheme iteratively loads parameters step by step.
The iterator provides data-in selections. The y is the Left part of fitting the equations. The Transformation corresponds to the nonlinear function f, and generates the Left part of the fitting model. The Processor coordinates all the actions and corrects the Regression parameters. Following picture represents this model.
The Data object is the same as in 6.1.1, the Iterator is of the following DataSetIterator
type.
/// <summary> /// Iterator obtained from data set /// </summary> [Serializable()] public class DataSetIterator : CategoryObject, ISerializable, IIterator, IDataSetConsumer, IMeasurements { // Code is contained in the source file }
This type implements the IIterator
interface.
/// <summary> /// Iterator /// </summary> public interface IIterator { /// <summary> /// Resets itself /// </summary> void Reset(); /// <summary> /// Go to next and false otherwise /// </summary> /// <returns>True if has next and</returns> bool Next(); }
Implemetation of this interface is presented below.
/// <summary> /// Reference to a table row /// </summary> protected DataRow[] rowreference = new DataRow[] { null }; #region IIterator Members void IIterator.Reset() { current = 0; rowm[0] = table.Rows[0]; } bool IIterator.Next() { if (table == null) { return false; } ++current; if (current >= table.Rows.Count) { return false; } row = table.Rows[current]; rowreference[0] = row; return true; } #endregion
So this class iterates rows of data table step by step, and references to different rows. A reference to a row is implemented as an array (See here). The reference is used for implementation of IMeasure
interface by following way.
/// <summary> /// Measurement from data row /// </summary> class RowMeasure : IMeasure { string name; object type; DataRow[] row; int ordinal; Func<object> par; internal RowMeasure(string name, object type, DataRow[] row, int ordinal) { this.name = name; this.type = type; this.row = row; this.ordinal = ordinal; par = Get; } #region IMeasure Members Func<object> IMeasure.Parameter { get { return par; } } string IMeasure.Name { get { return name; } } object IMeasure.Type { get { return type; } } #endregion #region Private Members object Get() { return row[0][ordinal]; } #endregion
Above code means that elements of rows are used as measurements. The Iterator as IMeasurement
is used by Formula, properties of Formula are presented below.
Formulas of this sample coincide with formulas of previous sample, however types of parametres do not coincide. Here x, y and z are parameters of the Double
type, in previous sample they are pararaters of Double[1000]
"type". So these formulas return two parameters of the Double
type, previous formulas return two parameters of the Double[1000]
"type". Properties of the Processor object are presented below:
Above picture means that we would like define values a, b, c, d, d of Formula such that Formula_1 (resp. Formula_2 of the Formula approximate parameters f1 (resp. f2) of the Iterator. The result coincides with result of the previous sample, however it does not require a large random-access memory.
6.2 2D Charts
Following picture contains the Hertzsprung-Russell-Diagram, obtained from http://www.astro.uni-bonn.de/~deboer/sterne/hrdtxt.html .
Above diagram is similar to a diagram provided by my framework.
The Data performs SQL query to the Hipparcos database.
The Formula provides necessary calculations, in particular it calculates absolute magnitudes of stars. The Color is an object of DrawSeries
type. Properties of the Color are displayed below.
It means that X, Y coordinates of 2D indication correspond to Formula_2, Formula_1 of the Formula respectively. Colors and sizes of indicated points are defined by Forlula_3, ..., Forlula_6 of the Formula. The database can be replaced with Internet data, following picture displays application of NASA/IPAC Extragalactic Database
However if data table contains a lot of rows then application of Internet databases becomes problematic. 2D indication picture can be extended by the following way.
Now we have a regression besides indication. The Filer is an object of the FormulaFilterIterator
class which implements the IIterator
. This object is intended for data filtering. Properties of this object is contained below.
Filter condition is a logical conjunction of following 3 conditions:
- x>ay means that the error of parallax is more than three times exceeding the parallax error;
- z< b means parallax does not exceed b(b=1000);
- |c + v - kz|< d means that the object belogs to admissible domain as it is displayed below.
The stars which belong to the admissible domain are approximated by polynomial of degree 3 as it is displayed below.
The Regression supplies necessary formulas.
- Download generator of Hipparcos and Tycho star catalogue - 134 KB
- Download sample of 2D charts - 453 KB
6.3 3D Charts + 6D Kinematics
Following picture accomplished by movie displays 3D chart with 6D kinematics.
The Stars is an object of PositionCollectionData
type, properties of this object are displayed below.
These properties are similar to properties of 2D chart. Other componetns supply following operations:
- Calculation of 3D positions from right ascension, declination and parallax. Common calculation issues are described here.
- Calculation of colors and positions of 3D points.
- Calculation of 6D motion parameters.
- 6D kinematics, which is described here.
- 3D graphics, which is described here.
- Download generator of Hipparcos and Tycho star catalogue - 134 KB
- Download sample of 3D charts - 137 KB
6.4 Databases + Digital image processing + Statistics.
In my former article I described an application of 2D image as statistical data set. Comarison from this data set and database data set enables us estimate physical parameters. Following picture explains this sample.
This sample can be used for navigation. Algorithm of this task is presented below:
Left part of this scheme contains image processing. Right part represents usage of star catalogue. Bridge compares both of them and in result we have parameters of spacecraft orientation. Let us consider details.
6.4.1 Image processing
Suppose that equipment provides following celestial image:
This image contains interfering information. We need filtration for its exclusion. Nonlocal digital image processing is being used for this purpose. Sheme of this processing is presented below:
This scheme contains Initial image (Source image) obtained by equipment. Little squares provides necessary math. It result we have Filtered image (Filtration result). Both images are presented below:
Following picture explains filtration algorithm:
If we have 9 closed white pixels then we replace it by one black pixel. Every other pixels are white. Result of filtration enables us to obtain X and Y coordinates of black pixels. Then this numbers will be compared with star catalogue. The Stat component extracts this numbers from Filtered image.
6.4.2 Application the of database
Star catalogue is stored in database. Necessary information can be extracted by SQL query:
Query statement is presented below:
++++++++++++++
SELECT RAdeg, DEdeg FROM hip_main WHERE RAdeg > @RAMIN AND RAdeg < @RAMAX AND DEdeg > @DEMIN AND DEDeg < @DEMAX AND BTmag > @BTMIN ORDER BY RAdeg
++++++++++++++
This statement has following meaning. First of all we consider limited area of sky. Declination and right ascension belong to small intervals. Secondly we consider stars only such that magnitudes exceed defined constant (in this sample the constant is equal to 9). Query result provides following chart:
We would like compare this chart with filtered image. This operation requires a set of math transformations. Full scheme of these transformations is presented below:
Essential feature of these transformations is Euclidean transformation:
Parameters a, b, and
are unknown. Comparison of star catalogue and filtered image
enable us to define these parameters. Using these parameters we can define
orientation of spacecraft.