Integration: Kinematics + Digital Image Processing + 3D Graphics

• Download source code - 3.7 MB
• Download Containers.zip 362.7 KB
• Download "Cube" sample 3.6 MB
• Download sample with processing of one image - 1.5 MB 
• Download sample with processing of two images - 3.6 MB

 

1. Introduction

This article promotes integration idea as well as a lot of my other articles. This idea is illustrated by following problem. We would like visualize 3D object which has texture obtained by digital image processing.

2. Data exchange architecture

2.1 Basic interfaces

All problems of article's title need data exchange. Data exchange architecture has four basic interfaces. First interface is elementary unit of data exchange.

     /// <summary>
    /// Elementary unit of data exchange
    /// </summary>
    public interface IMeasure
    {
        /// <summary>
        /// Function which returns unit of data
        /// </summary>
        Func<object> Parameter
        {
            get;
        }

        /// <summary>
        /// The name of data unit
        /// </summary>
        string Name
        {
            get;
        }

        /// <summary>
        /// Type of parameter
        /// </summary>
        object Type
        {
            get;
        }
    }

Members of this interface are presented in following table.

N	Member name	Meaning
1	Parameter	Function which returns unit of data
2	Name	The name of data unit
3	Type	Type of parameter

The Parameter function can return any object. The Name property enables us to link input parameters to output ones. The Type property contains Meta information about unit. This property is not necessary variable of System.Type type. Following table is a mapping between variables and their "types".

N	Type of variable	Value of Type property
1	string	string s = ""
2	bool	bool b = false
3	double	double d = 0
4	float	float f = 0
5	byte	byte b = 0x0
6	sbyte	sbyte s = 0x0
7	short	short s = 0
8	short	ushort u = 0
9	int	int i = 0
10	uint	uint u = 0
11	long	long l = 0
12	ulong	ulong u = 0
13	Array	ArrayReturnType a = (depends on type and length of array)

If elementary unit parameter is array then "type" of unit is ArrayReturnType variable. The lArrayReturnType code is presented below.

    /// <summary>
    /// Return type of array
    /// </summary>
    public class ArrayReturnType
    {

        #region Fields

        object elementType;
        int[] dimension;
        bool isObjectType;


        #endregion

        /// <summary>
        /// Constructor
        /// </summary>
        /// <param name="elementType">Type of element</param>
        /// <param name="dimension">Dimension</param>
        /// <param name="objectType">The "is object" sign</param>
        public ArrayReturnType(object elementType, int[] dimension, bool objectType)
        {
            this.elementType = elementType;
            this.dimension = dimension;
            this.isObjectType = objectType;
        }

        /// <summary>
        /// The "is object" sign
        /// </summary>
        public bool IsObjectType
        {
            get
            {
                return isObjectType;
            }
        }

        /// <summary>
        /// Dimension
        /// </summary>
        public int[] Dimension
        {
            get
            {
                return dimension;
            }
        }

        /// <summary>
        /// Type of element
        /// </summary>
        public object ElementType
        {
            get
            {
                return elementType;
            }
        }

        /// <summary>
        /// Overriden Equals
        /// </summary>
        /// <param name="obj">Compared obje</param>
        /// <returns>True if equal</returns>
        public override bool Equals(object obj)
        {
            if (!(obj is ArrayReturnType))
            {
                return false;
            }
            ArrayReturnType at = obj as ArrayReturnType;
            if (!at.elementType.Equals(elementType))
            {
                return false;
            }
            if (at.dimension.Length != dimension.Length)
            {
                return false;
            }
            for (int i = 0; i < dimension.Length; i++)
            {
                if (dimension[i] != at.dimension[i])
                {
                    return false;
                }
            }
            return true;
        }

        /// <summary>
        /// Overriden
        /// </summary>
        /// <returns>Hash code</returns>
        public override int GetHashCode()
        {
            return dimension.Length * dimension[0];
        }

        /// <summary>
        /// Checks equality of imension
        /// </summary>
        /// <param name="type">Type</param>
        /// <returns>True in case of equal dimesion and false otherwise</returns>
        public bool HasEqualDimension(ArrayReturnType type)
        {
            if (type.dimension.Length != dimension.Length)
            {
                return false;
            }
            for (int i = 0; i < dimension.Length; i++)
            {
                if (dimension[i] != type.dimension[i])
                {
                    return false;
                }
            }
            return true;
        }

        /// <summary>
        /// Gets base type
        /// </summary>
        /// <param name="o">The object</param>
        /// <returns>Object's base type</returns>
        static public object GetBaseType(object o)
        {
            if (o is ArrayReturnType)
            {
                ArrayReturnType rt = o as ArrayReturnType;
                return rt.elementType;
            }
            return o;
        }
    }

Following table contains samples of variables with their Type property.

N	Variable	Value of Type property	Comment
1	double[] d = new double[3];	new ArrayReturnType((double)0, new int[]{3}, false);
2	object[] o = new object[2];	new ArrayReturnType((double)0, new int[]{2}, true);	Every element of o is double
3	bool[,] b = new object[2,3];	new ArrayReturnType(false, new int[]{2, 3}, false);
4	string[,,] s = new string[5,2,3];	new ArrayReturnType("", new int[]{5, 2, 3}, false);

Set of supported types of parameters can be extended.

Second interface is implemented by all providers of data.

     /// <summary>
    /// Data provider
    /// </summary>
    public interface IMeasurements
    {
        /// <summary>
        /// The count of data units
        /// </summary>
        int Count
        {
            get;
        }

        /// <summary>
        /// Gets number - th unit of data
        /// </summary>
        IMeasure this[int number]
        {
            get;
        }

        /// <summary>
        /// Updates data
        /// </summary>
        void UpdateMeasurements();


        /// <summary>
        /// Shows, weather the object is updated
        /// </summary>
        bool IsUpdated
        {
            get;
            set;
        }
    }

Following table explains meaning of this interface members

N	Member name	Meaning	Comment
1	Count	The count of data units
2	this[int number]	Gets numberth unit of data
3	UpdateMeasurements()	Updates data	Data can be updated by the time
4	IsUpdated	Shows, weather the object is updated	This property is used for avoiding multiple update

All consumers of data implement following interface.

     /// <summary>
    /// Consumer of data
    /// </summary>
    public interface IDataConsumer
    {
 
        /// <summary>
        /// Adds data provider 
        /// </summary>
        /// <param name="measurements">Provider to add</param>
        void Add(IMeasurements measurements);

        /// <summary>
        /// Removes data provider
        /// </summary>
        /// <param name="measurements">Provider to remove</param>
        void Remove(IMeasurements measurements);

        /// <summary>
        /// Updates data of data providers
        /// </summary>
        void UpdateChildrenData();

        /// <summary>
        /// Count of providers
        /// </summary>
        int Count
        {
            get;
        }

        /// <summary>
        /// Access to n - th provider
        /// </summary>
        IMeasurements this[int number]
        {
            get;
        }

        /// <summary>
        /// Resets measurements
        /// </summary>
        void Reset();

        /// <summary>
        /// Change Input event
        /// </summary>
        event Action OnChangeInput;
    }

Following interface is auxiliary. 

    /// <summary>
    /// Collection on named data units
    /// </summary>
    public interface IAlias : IAliasBase
    {
        /// <summary>
        /// Names of all data units
        /// </summary>
        IList<string> AliasNames
        {
            get;
        }

        /// <summary>
        /// Access to data unit by name
        /// </summary>
        object this[string name]
        {
            get;
            set;
        }

        /// <summary>
        /// Gets unit type
        /// </summary>
        /// <param name="name">Unit name</param>
        /// <returns>Type of unit</returns>
        object GetType(string name);

    }

Following class links data consumers to data providers.

    /// <summary>
    /// The link between data provider and data consumer
    /// </summary>
    [Serializable()]
    public class DataLink : ICategoryArrow, ISerializable,
        IRemovableObject, IDataLinkFactory
    {

        #region Fields

        /// <summary>
        /// Error message
        /// </summary>
        public static readonly string SetProviderBefore = "You should create measurements source before consumer";

        /// <summary>
        /// DataLink checker
        /// </summary>
        private static Action<DataLink> checker;

        /// <summary>
        /// The source of this arrow
        /// </summary>
        private IDataConsumer source;

        /// <summary>
        /// The target of this arrow
        /// </summary>
        private IMeasurements target;

        /// <summary>
        /// Auxiliary field
        /// </summary>
        private int a = 0;

        /// <summary>
        /// Linked object
        /// </summary>
        protected object obj;

        /// <summary>
        /// Data link factory
        /// </summary>
        private static IDataLinkFactory dataLinkFactory = new DataLink();


        #endregion

        #region Ctor

        /// <summary>
        /// Default constructor
        /// </summary>
        public DataLink()
        {
        }

        /// <summary>
        /// Deserialization constructor
        /// </summary>
        /// <param name="info">Serialization info</param>
        /// <param name="context">Streaming context</param>
        public DataLink(SerializationInfo info, StreamingContext context)
        {
            a = (int)info.GetValue("A", typeof(int));
        }

        #endregion

        #region ISerializable Members

        /// <summary>
        /// ISerializable interface implementation
        /// </summary>
        /// <param name="info">Serialization info</param>
        /// <param name="context">Streaming context</param>
        public void GetObjectData(SerializationInfo info, StreamingContext context)
        {
        }


        #endregion

        #region ICategoryArrow Members

        /// <summary>
        /// The source of this arrow
        /// </summary>
        public ICategoryObject Source
        {
            set
            {
                if (source != null)
                {
                    throw new Exception();
                }
                IDataLinkFactory f = this;
                source = f.GetConsumer(value);
            }

            get
            {
                return source as ICategoryObject;
            }
        }

        /// <summary>
        /// The target of this arrow
        /// </summary>
        public ICategoryObject Target
        {
            get
            {
                return target as ICategoryObject;
            }
            set
            {
                if (target != null)
                {
                    throw new Exception();
                }
                IDataLinkFactory f = this;
                 bool check = true;
                IAssociatedObject s = source as IAssociatedObject;
                if (s.Object != null & value.Object != null)
                {
                    if (check)
                    {
                        INamedComponent ns = s.Object as INamedComponent;
                        INamedComponent nt = value.Object as INamedComponent;
                        if (nt != null & ns != null)
                        {
                            if (PureDesktopPeer.GetDifference(nt, ns) >= 0)
                            {
                                throw new Exception(SetProviderBefore);
                            }
                        }
                    }
                    target = t;
                    source.Add(target);
                }
                if (!check)
                {
                    return;
                }
                try
                {
                    if (checker != null)
                    {
                        checker(this);
                    }
                }
                catch (Exception e)
                {
                    e.ShowError(10);
                    source.Remove(target);
                    throw e;
                }
            }
        }

        /// <summary>
        /// The "is monomorhpism" sign
        /// </summary>
        public bool IsMonomorphism
        {
            get
            {
                return false;
            }
        }

        /// <summary>
        /// The "is epimorhpism" sign
        /// </summary>
        public bool IsEpimorphism
        {
            get
            {
                return false;
            }
        }


        /// <summary>
        /// The "is isomorhpism" sign
        /// </summary>
        public bool IsIsomorphism
        {
            get
            {
                return false;
            }
        }


        /// <summary>
        /// Composes this arrow "f" with next arrow "g" 
        /// </summary>
        /// <param name="category"> The category of arrow</param>
        /// <param name="next"> The next arrow "g" </param>
        /// <returns>Composition "fg" </returns>
        public ICategoryArrow Compose(ICategory category, ICategoryArrow next)
        {
            return null;
        }

        #endregion

        #region IAssociatedObject Members

        /// <summary>
        /// Associated object
        /// </summary>
        public object Object
        {
            get
            {
                return obj;
            }
            set
            {
                obj = value;
            }
        }

        #endregion

        #region IRemovableObject Members

        /// <summary>
        /// The post remove operation
        /// </summary>
        public void RemoveObject()
        {
            if (source == null | target == null)
            {
                return;
            }
            source.Remove(target);
        }

        #endregion

        #region IDataLinkFactory Members

        IDataConsumer IDataLinkFactory.GetConsumer(ICategoryObject source)
        {
            IAssociatedObject ao = source;
            object o = ao.Object;
            if (o is INamedComponent)
            {
                IDataConsumer dcl = null;
                INamedComponent comp = o as INamedComponent;
                IDesktop desktop = comp.Root.Desktop;
                desktop.ForEach<DataLink>(delegate(DataLink dl)
                {
                    if (dcl != null)
                    {
                        return;
                    }
                    object dt = dl.Source;
                    if (dt is IAssociatedObject)
                    {
                        IAssociatedObject aot = dt as IAssociatedObject;
                        if (aot.Object == o)
                        {
                            dcl = dl.source as IDataConsumer;
                        }
                    }
                });
                if (dcl != null)
                {
                    return dcl;
                }
            }

            IDataConsumer dc = DataConsumerWrapper.Create(source);
            if (dc == null)
            {
                CategoryException.ThrowIllegalTargetException();
            }
            return dc;
        }

        IMeasurements IDataLinkFactory.GetMeasurements(ICategoryObject target)
        {
            IAssociatedObject ao = target;
            object o = ao.Object;
            if (o is INamedComponent)
            {
                IMeasurements ml = null;
                INamedComponent comp = o as INamedComponent;
                IDesktop d = null;
                INamedComponent r = comp.Root;
                if (r != null)
                {
                    d = r.Desktop;
                }
                else
                {
                    d = comp.Desktop;
                }
                if (d != null)
                {
                    d.ForEach<DataLink>(delegate(DataLink dl)
                    {
                        if (ml != null)
                        {
                            return;
                        }
                        object dt = dl.Target;
                        if (dt is IAssociatedObject)
                        {
                            IAssociatedObject aot = dt as IAssociatedObject;
                            if (aot.Object == o)
                            {
                                ml = dl.Target as IMeasurements;
                            }
                        }
                    });
                    if (ml != null)
                    {
                        return ml;
                    }
                }
            }
            IMeasurements m = MeasurementsWrapper.Create(target);
            if (m == null)
            {
                CategoryException.ThrowIllegalTargetException();
            }
            return m;
        }

        #endregion

        #region Public Members

        /// <summary>
        /// Checker of data link
        /// </summary>
        public static Action<DataLink> Checker
        {
            set
            {
                checker = value;
            }
        }

        /// <summary>
        /// Data link factory
        /// </summary>
        public static IDataLinkFactory DataLinkFactory
        {
            get
            {
                return dataLinkFactory;
            }
            set
            {
                dataLinkFactory = value;
            }
        }


        /// <summary>
        /// Measurements provider
        /// </summary>
        public IMeasurements Measurements
        {
            get
            {
                return target;
            }
        }

        #endregion
    }

implies execution of following operator

   		consumer.Add(provider);    			// Consumer adds a provider

.

Indeed above operators are never written explicitly, but they are implied by following graphical designer.

The Link arrow corresponds to DataLink object. Source (Chart) (resp. target (Formula)) object is object which implements IDataConsumer (resp. IMeasurements) interface. Graphical setting of Link arrow implies execution of next operator:

   		consumer.Add(provider);    			// Consumer adds a provider

Deleting of the arrow implies execution of next operator:

   		consumer.Remove(provider);    		// Consumer removes a provider

2.2 Sample

Following sample exhibits data exchange.

This sample contains three objects and three DataLink arrows. Following table contains types of objects.

N	Object	Type	Implemented interfaces
1	Formula	VectorFormulaConsumer	IDataConsumer, IMeasurements, IAlias
2	Differential equation	DifferentialEquationSolver	IDataConsumer, IMeasurements, IAlias
3	Chart	DataConsumer	IDataConsumer

VectorFormulaConsumer and DifferentialEquationSolver implement both IDataConsumer and IMeasurements interfaces. So these objects can be both sources and targets of DataLink arrow. But DataConsumer object can be source only. The Formula object has following properties

Variables a and b are marked. It means that these variables correspond to IAlias interface. Following code snippet explains this circumstance.

        	   VectorFormulaConsumer formula = ...;
            IAlias alias = formula;
            IList<string> names = alias.AliasNames;     // names[0] = "a", names[1] = "b";
            double a = (double)alias["a"];              // a = 1;
            double b = (double)alias["b"];              // b = 2;
            object ta = alias.GetType("a");             // ta = (double)0;
            object tb = alias.GetType("b");             // tb = (double)0;

The t variable corresponds to virtual time.

The Differential equation object has following properties

This object performs solution of following system of ordinary differential equations

 

The Chart object indicates time dependencies. Its properties are clear.

 

3. Kinematics architecture

3.1 Main idea

Frame of reference is a basic notion of kinematics. Presented here architecture uses relative frames of reference, those have forest structure. Example of such structure is presented in the following diagram:

 

There exists a zero frame of reference. All root nodes of the forest correspond to this frame. 6D positions of other frames (nodes) are relative to 6D positions of those parents. It means that 6D position of frame 1.1.2 is relative to 6D position of frame 1.1 etc. This architecture enables us easily simulate a lot of different useful situations. For example we can install a set of virtual cameras on air(space)craft. Also it is easy to visualize complicated motion of helicopter rotor (slow motion) a plane with swing-wing or thrust vector control.

3.2 Basic classes and interfaces 

Following two interfaces correspond to 3D position and 3D orientation respectively

     /// <summary>
    /// 3D Position
    /// </summary>
    public interface IPosition
    {

        /// <summary>
        /// Absolute position coordinates
        /// </summary>
        double[] Position
        {
            get;
        }

        /// <summary>
        /// Parent frame
        /// </summary>
        IReferenceFrame Parent
        {
            get;
            set;
        }

        /// <summary>
        /// Position parameters
        /// </summary>
        object Parameters
        {
            get;
            set;
        }

        /// <summary>
        /// Updates itself
        /// </summary>
        void Update();
    }

   /// <summary>
    /// Object with orientation
    /// </summary>
    public interface IOrientation
    {
        /// <summary>
        /// Orientation quaternion
        /// </summary>
        double[] Quaternion
        {
            get;
        }

        /// <summary>
        /// Orientation matrix
        /// </summary>
        double[,] Matrix
        {
            get;
        }
    }

The ReferenceFrame class implements both IPosition and IOrientation. Its code is very long and is contained is source files. Following interface supplies relative motion paradigm.

    /// <summary>
    /// Reference frame holder
    /// </summary>
    public interface IReferenceFrame : IPosition
    {
        /// <summary>
        /// Own frame
        /// </summary>
        ReferenceFrame Own
        {
            get;
        }

        /// <summary>
        /// Children objects
        /// </summary>
        List<IPosition> Children
        {
            get;
        }

    }

The Parent property of IReferenceFrame interface is the parent frame of reference. Motion of point is relative with respect to parent frame. The Children property of IReferenceFrame is collection of children points. Following class links child to parent.

     /// <summary>
    /// Link of relative frame
    /// </summary>
  [Serializable()]
    public class ReferenceFrameArrow : CategoryArrow, ISerializable, IRemovableObject
    {
        #region Fields
        
        IPosition source;
        
        IReferenceFrame target;
         
        #endregion

        #region Constructors

        /// <summary>
        /// Default constructor
        /// </summary>
        public ReferenceFrameArrow()
        {
        }


        /// <summary>
        /// Deserialization constructor
        /// </summary>
        /// <param name="info">Serialization info</param>
        /// <param name="context">Streaming context</param>
        protected ReferenceFrameArrow(SerializationInfo info, StreamingContext context)
        {
        }

        #endregion

        #region ISerializable Members

        void ISerializable.GetObjectData(SerializationInfo info, StreamingContext context)
        {
        }

        #endregion

        #region ICategoryArrow Members

        /// <summary>
        /// The source of this arrow
        /// </summary>
        public override ICategoryObject Source
        {
            get
            {
                return source as ICategoryObject;
            }
            set
            {
                IPosition position = value.GetSource<IPosition>();
                if (position.Parent != null)
                {
                    throw new CategoryException("Root", this);
                }
                source = position;
            }
        }

        /// <summary>
        /// The target of this arrow
        /// </summary>
        public override ICategoryObject Target
        {
            get
            {
                return target as ICategoryObject;
            }
            set
            {
                IReferenceFrame rf = value.GetTarget<IReferenceFrame>();
                IAssociatedObject sa = source as IAssociatedObject;
                IAssociatedObject ta = value as IAssociatedObject;
                INamedComponent ns = sa.Object as INamedComponent;
                INamedComponent nt = ta.Object as INamedComponent;
                target = rf;
                source.Parent = target;
                target.Children.Add(source);
            }
        }


        #endregion

        #region IRemovableObject Members

        void IRemovableObject.RemoveObject()
        {
            source.Parent = null;
            if (target != null)
            {
                target.Children.Remove(source);
            }
        }

        #endregion

        #region Specific Members

        /// <summary>
        /// Preparation operation
        /// </summary>
        /// <param name="collection">Desktop</param>
        /// <returns>List of position objects</returns>
        static public List<IPosition> Prepare(IComponentCollection collection)
        {
            List<IPosition> frames = new List<IPosition>();
            if (collection == null)
            {
                return frames;
            }
            IEnumerable<object> c = collection.AllComponents;
            foreach (object o in c)
            {
                if (!(o is IObjectLabel))
                {
                    continue;
                }
                IObjectLabel lab = o as IObjectLabel;
                ICategoryObject co = lab.Object;
                if (!(co is IReferenceFrame))
                {
                    if (co is IPosition)
                    {
                        IPosition p = co as IPosition;
                        if (p.Parent == null)
                        {
                            frames.Add(p);
                        }
                    }
                    continue;
                }
                IReferenceFrame f = co as IReferenceFrame;
                if (f.Parent != null)
                {
                    continue;
                }
                prepare(f, frames);
            }
            return frames;
        }

        /// <summary>
        /// Updates frames
        /// </summary>
        /// <param name="frames">List of frames</param>
        public static void Update(List<IPosition> frames)
        {
            foreach (IPosition frame in frames)
            {
                frame.Update();
            }
        }


        private static void prepare(IReferenceFrame frame, List<IPosition> frames)
        {
            List<IPosition> children = frame.Children;
            frames.Add(frame);
            foreach (IPosition p in children)
            {
                if (frames.Contains(p))
                {
                    continue;
                }
                if (p is IReferenceFrame)
                {
                    IReferenceFrame f = p as IReferenceFrame;
                    prepare(f, frames);
                }
                else
                {
                    frames.Add(p);
                }
            }
        }

        #endregion
    }

Source of this arrow is IPoint object, target is a IReferenceFrame one. Setting of ReferenceFrameArrow arrow

             IPosition position = ...;
            IReferenceFrame frame = ...;
            ReferenceFrameArrow link = new ReferenceFrameArrow();
            link.Source = position as ICategoryObject;
            link.Target = position as ICategoryObject;

implies execution of following operators

           position.Parent = frame;
            frame.Children.Add(position);  

.

Setting of all arrows is provided by graphical designer.

 

This picture has three links L1, L2 and L3 of ReferenceFrameArrow type and four objects

N	Name	Type	Implemented interfaces	Parent
1	Base frame	RigidReferenceFrame	IPosition, IReferenceFrame	null
2	Frame 1	ReferenceFrameData	IPosition, IReferenceFrame, IDataConsumer	Base frame
3	Frame 2	ReferenceFrameData	IPosition, IReferenceFrame, IDataConsumer	Frame 1
4	Camera frame	RigidReferenceFrame	IPosition, IReferenceFrame	Frame 2
5	Camera	WpfCamera	IPosition	Camera frame

Since parent of Base frame is null motion of this frame is regarded as motion with respect to zero frame of reference. The RigidReferenceFrame class corresponds to frames with constant relative coordinates and orientation. The ReferenceFrameData class corresponds to moving frames of reference. This class implements IDataConsumer interface for interoperability with data exchange.

3.3 Interoperability with data exchange

The ReferenceFrameData implements IDataConsumer interface. It uses external data as parameters of relative motions. Following picture represents application of ReferenceFrameData

The Motion Parameters object calculates motion parameters by following way:

 

This object as IMeasurements is linked to Motion Frame as IDataConsumer by DataLink D. The Motion Frame object uses output data of Motion Parameters.

N	Motion parameter of Motion Frame	Output parameter of Motion Parameters
1	Relative x - coordinate	Formula_3
2	Relative y - coordinate	Formula_4
3	Relative z - coordinate	Formula_4
4	Q0 component of relative orientation quaternion	Formula_1
5	Q1 component of relative orientation quaternion	Formula_2
6	Q2 component of relative orientation quaternion	Formula_4
7	Q3 component of relative orientation quaternion	Formula_4

Besides IDataConsumer kinematic architecture contains RelativeMeasurements which implements IMeasurements interface. Type of above Relative object is RelativeMeasurements. The R1, R2 arrow between Relative and frames mean that Relative provides parameters of Motion Frame relative motion with respect to Base frame. Since Relative implements IMeasurements interface it can be connected by DataLink to IDataConsumer

.

 

Here Chart object indicates relative x, y and z coordinates.

4. Digital image processing architecture

4.1 Main interfaces and classes

Roughly speaking digital image processing obtains one image from other ones. So architecture contains provides and consumers of images. Following interface is provider of images.

	/// <summary>
	/// Provider of image
	/// </summary>
	public interface IBitmapProvider
	{
		/// <summary>
		/// Bitmap
		/// </summary>
		Bitmap Bitmap
		{
			get;
		}
	}

Following interface is consumer of image.

	/// <summary>
	/// Consumer of image
	/// </summary>
	public interface IBitmapConsumer
	{
		/// <summary>
		/// Procesess image
		/// </summary>
		void Process();

		/// <summary>
		/// Providers
		/// </summary>
		IEnumerable<IBitmapProvider> Providers
		{
			get;
		}

        		/// <summary>
        		/// Adds a provider
        		/// </summary>
        		/// <param name="provider">The provider</param>
        		void Add(IBitmapProvider provider);

        		/// <summary>
        		/// Removes a provider
        		/// </summary>
        		/// <param name="provider">The provider</param>
        		void Remove(IBitmapProvider provider);
	}

Following class is link between IBitmapProvider and IBitmapConsumer

    /// <summary>
    /// Link between bitmap consumer and bitmap provider
    /// </summary>
    [Serializable()]
    public class BitmapConsumerLink : ICategoryArrow, IRemovableObject, ISerializable
    {
        #region Fields

        /// <summary>
        /// Error message
        /// </summary>
        static public readonly string ProviderExists = "Bitmap provider already exists";
        
        /// <summary>
        /// Error message
        /// </summary>
        public static readonly string SetProviderBefore = "You should create bitmap provider before consumer";

        /// <summary>
        /// Associated object
        /// </summary>
        private object obj;

        /// <summary>
        /// Auxiliary variable
        /// </summary>
        private int a = 0;

        /// <summary>
        /// Source
        /// </summary>
        private IBitmapConsumer source;

        /// <summary>
        /// Target
        /// </summary>
        private IBitmapProvider target;

        #endregion

        #region Constructors
        public BitmapConsumerLink()
        {
        }

        public BitmapConsumerLink(SerializationInfo info, StreamingContext context)
        {
            info.GetValue("A", typeof(int));
        }

        #endregion

        #region ICategoryArrow Members

        public ICategoryObject Source
        {
            get
            {
                return source as ICategoryObject;
            }
            set
            {
                source = value.GetObject<IBitmapConsumer>();
            }
        }

        public ICategoryObject Target
        {
            get
            {
                return target as ICategoryObject;
            }
            set
            {
                target = value.GetObject<IBitmapProvider>();
                source.Add(target);
            }
        }

        public bool IsMonomorphism
        {
            get
            {
                return false;
            }
        }

        public bool IsEpimorphism
        {
            get
            {
                return false;
            }
        }

        public bool IsIsomorphism
        {
            get
            {
                return false;
            }
        }

        public ICategoryArrow Compose(ICategory category, ICategoryArrow next)
        {
            return null;
        }

        #endregion

        #region IAssociatedObject Members

        public object Object
        {
            get
            {
                return obj;
            }
            set
            {
                obj = value;
            }
        }

        #endregion

        #region IRemovableObject Members

        public void RemoveObject()
        {
            if (source != null & target != null)
            {
                source.Remove(target);
            }
        }

        #endregion

        #region ISerializable Members

        public void GetObjectData(SerializationInfo info, StreamingContext context)
        {
        }

        #endregion

        #region Specific Members

        /// <summary>
        /// Updates consumer
        /// </summary>
        /// <param name="consumer">Consumer</param>
        public static void Update(IBitmapConsumer consumer)
        {
            IEnumerable<IBitmapProvider> providers = consumer.Providers;
            foreach (IBitmapProvider provider in providers)
            {
                if (provider is IBitmapConsumer)
                {
                    IBitmapConsumer c = provider as IBitmapConsumer;
                    Update(c);
                }
            }
            consumer.Process();
        }

        /// <summary>
        /// Gets provider for consumer
        /// </summary>
        /// <param name="provider">Provider</param>
        /// <param name="consumer">Consumer</param>
        /// <param name="mutipleProviders">The multiple providers flag</param>
        /// <returns>The provider</returns>
        public static IBitmapProvider GetProvider(IBitmapProvider provider, IBitmapConsumer consumer, bool mutipleProviders)
        {
            if (provider == null)
            {
                return null;
            }
            ICategoryObject t = provider as ICategoryObject;
            ICategoryObject s = consumer as ICategoryObject;
            if (s.Object != null & t.Object != null)
            {
                INamedComponent ns = s.Object as INamedComponent;
                INamedComponent nt = t.Object as INamedComponent;
                if (nt != null & ns != null)
                {
                    if (nt.Desktop == ns.Desktop)
                    {
                        if (nt.Ord >= ns.Ord)
                        {
                            throw new Exception(SetProviderBefore);
                        }
                    }
                    else
                    {
                        if (nt.Root.Ord >= ns.Root.Ord)
                        {
                            throw new Exception(SetProviderBefore);
                        }
                    }
                }
            }
            return provider;
        }

        #endregion

    } 

Setting of BitmapConsumerLink arrow

          IBitmapProvider provider = ...;
            IBitmapConsumer consumer = ...;
            BitmapConsumerLink link = new BitmapConsumerLink();
            link.Source = provider as ICategoryObject;
            link.Target = consumer as ICategoryObject;
    

implies execution of following operator

consumer.Add(provider);

. However above operations are performed by graphics designer.

 

Here Lady Rose and Lady Blue are IBitmapProvider and IBitmapConsumer respectively. These objects are linked by BitmapConsumerLink

4.2 Interoperability with data exchange

4.2.1 Processing of one bitmap

Digital image processing has classes which implement IDataConsumer and/or IMeasurements interfaces. Let us consider local digital filtration sample.

Above picture contains following objects.

N	Object name	Type	Implemented interfaces
1	Earth	SourceBitmap	IBitmapProvider
2	Result of processing	BitmapTransformer	IBitmapProvider, IBitmapConsumer, IDataConsumer
3	Formulae	VectorFormulaConsumer	IMeasurements, IAlias

The Result of processing object as IBitmapConsumer is connected to Earth as IBitmapProvider by BitmapConsumerLink. The Result of processing object as IDataConsumer is connected to Formulae as IMeasurements by DataLink. Any object of SourceBitmap class stores just stores image in memory. Main members of this class are presented below.

         #region Fields

        /// <summary>
        /// Bitmap
        /// </summary>
        protected Bitmap bitmap;

        #endregion

        #region Ctor

        /// <summary>
        /// Default constructor
        /// </summary>
        public SourceBitmap()
        {
        }

        /// <summary>
        /// Deserialization constructor
        /// </summary>
        /// <param name="info">Serialization info</param>
        /// <param name="context">Streaming context</param>
        public SourceBitmap(SerializationInfo info, StreamingContext context)
        {
            try
            {
                bitmap = (Bitmap)info.GetValue("Bitmap", typeof(Bitmap));
            }
            catch (Exception ex)
            {
                ex.ShowError(100);
            }
        }

        #endregion

       #region ISerializable Members


     /// <summary>
        /// ISerializable interface implementation
        /// </summary>
        /// <param name="info">Serialization info</param>
        /// <param name="context">Streaming context</param>
        public override void GetObjectData(SerializationInfo info, StreamingContext context)
        {
            info.AddValue("Bitmap", bitmap);
        }

       #region IBitmapProvider Members

        /// <summary>
        /// Bitmap
        /// </summary>
        Bitmap IBitmapProvider.Bitmap
        {
            get
            {
                return bitmap;
            }
        }

        #endregion

        /// <summary>
        /// Sets bitmap
        /// </summary>
        /// <param name="bitmap"></param>
        public void SetBitmap(Bitmap bitmap)
        {
            this.bitmap = bitmap;
        }

        // Other members ...

Business logic of SourceBitmap is very clear. This class has a field of Bitmap type. User can set value of this field? This field is serialized and is also used as Bitmap of IBitmapProvider interface. The Formulae object has following properties.

 

This formula calculates color from color of source bitmap. Parameters r, g and b correspond to red, green and blue color of source bitmap. If values of all colors exceed threshold value (a then formula returns x. Otherwise it returns y. Object Result of processing following properties.

 

These properties have following meaning. Object scans bitmap of provider object (Earth) and detects colors of pixels. It sets these objects as aliases of Formulae. Then it calculates Formulae and sets Formula_1 as red, green and blue color of result bitmap. Source bitmap and transformation result are presented below.

This algorithm is simplest rough algorithm of detection of snow mantle. C# explanation of this algorithm is presented below.

            //============    Formulae ==============

            VectorFormulaConsumer formulae = ...; // Formulae object
            
            IMeasurements measurements = formulae;

            IMeasure m = measurements[0];           // Formula_1
            IAlias alias = formulae;                //  Formulae object as IAlias
          
            // =========================================

            // =========== Source bitmap ===============
            SourceBitmap sb = ...;

            IBitmapProvider provider = sb;

            Bitmap source = provider.Bitmap;

            //===========================================

            // ============ Target bitmap ===============

            
            Bitmap target =     // Size of tagtet is equal to source one
                new Bitmap(source.Width, source.Height);

            for (int x = 0; x < source.Width; x++)       // x = coordinate of bitmap
            {
                for (int y = 0; y < source.Height; y++)  // y - coordinate of bitmap
                {
                    Color colorSource = source.GetPixel(x, y); // Color of source pixel
                    double r = (double)colorSource.R / 256;    // Scaling
                    double g = (double)colorSource.G / 256;
                    double b = (double)colorSource.B / 256;
                    alias["r"] = r;                             // Setting of aliases
                    alias["g"] = g;
                    alias["b"] = b;
                    measurements.UpdateMeasurements();          // Calculates all formulae
                    double cd = (double)m.Parameter();          // Formula_1
                    int cdi = (int)(cd * 256);                  // Inverse scaling
                    Color colorTarget = 
                        Color.FromArgb(cdi, cdi, cdi);          // Target color
                    target.SetPixel(x, y, colorTarget);         // Sets pixel to target bitmap
                }
            }

Note that colors of target can be different as it is shown in "Lady Blue" sample. Above code snippet does not present in my code, it is just clear explanation of algorithm. More information about this subject you can find in my article "Digital Image Processing".

4.2.2 Processing of two bitmaps 

Several tasks imply simultaneous processing of several images. For example clouds dynamics indication requires comparison of two or more images. Following picture represents comparison of two images

This picture has following objects

N	Object name	Type	Implemented interfaces	Comment
1	Picture 1	SourceBitmap	IBitmapProvider	First source image
2	Picture 2	SourceBitmap	IBitmapProvider	Second source image
3	P 1	BitmapColorTable	IBitmapConsumer, IDataConsumer, IMeasurements	Adapter object
4	P 2	BitmapColorTable	IBitmapConsumer, IDataConsumer, IMeasurements	Adapter object
5	Input	VectorFormulaConsumer	IMeasurements, IAlias	Digital image processing calculator
6	Result	VectorFormulaConsumer	IMeasurements, IAlias	Digital image processing calculator
7	Compare	BitmapTransformer	IBitmapProvider, IBitmapConsumer, IDataConsumer	Digital image processing result

The Compare object as IBitmapConsumer is connected to Picture 2 as IBitmapProvider by BitmapConsumerLink. It means that P 2 provides bitmap for Compare. According to BitmapTransformer implementation it means that size of Compare bitmap is the same as Picture 2 one. The Compare has following properties.

 

These properties have following meaning. The Compare object scans own bitmap and sets alias parameters as values of pixel coordinates. Then it sets Formula_1 value as red, green and blue color of image. Following code clarifies this algorithm.

             VectorFormulaConsumer Input = ...; // The "Input" object

            VectorFormulaConsumer Result = ...; // The "Result" object

            IAlias alias = Input;               // "Input" as IAlias

            IMeasurements measurements =        // "Result" as IMeasurements
                Result;

            IMeasure Formula_1 = measurements[0]; // "Formula_1"

            Bitmap Compare = ...;              // Bitmap of "Compare object"

            for (int x = 0; x < Compare.Width; x++)       // x = coordinate of bitmap
            {
                for (int y = 0; y < Compare.Height; y++)  // y - coordinate of bitmap
                {
                    alias["x"] = (double)x;               // Setting parameters
                    alias["y"] = (double)y;
                    measurements.UpdateMeasurements();    // Calculation
                    int color =                           // Color
                        (int)((double)Formula_1.Parameter() * 256);
                    Color c = Color.FromArgb(color, color, color);
                    Compare.SetPixel(x, y, c);            // Sets pixel color
                }
            }

Both P 1 and P 2 are objects of BitmapColorTable type. This type as IDataConsumer consumes coordinates pixel, and as IMeasurements provides RGB color parameters. Following picture

 

means that P 1 returns RGB parameters of Picture 1 bitmap pixel. Coordinate x (resp. y) of pixel equals to Formula_1, (resp. Formula_2) of Input. Properties of Input are presented below.

 

So parameter x (resp. y) of Input as IAlias equals to Formula_1 (resp. Formula_2) of the same object as IMeasurements. Thus input parameters of both P 1, P 2 are pixel coordinates of Compare bitmap. Properties of Result are presented below.

 

Parameter x (resp. y) of this object is Red parameter of P 1 (resp. P 2), i.e. red component of pixel of Picture 1 (resp. Picture 2). Formula_1 of Result is proportional to difference between red components of Picture 1 and Picture 2.

5. Architecture of 3D graphics

5.1 Main interfaces and classes

Main interfaces of 3D graphics are IVisible and IVisibleConsumer

.

    /// <summary>
    /// Object linked to position
    /// </summary>
    public interface IPositionObject
    {
        /// <summary>
        /// Linked position
        /// </summary>
        IPosition Position
        {
            get;
            set;
        }
    }

       /// <summary>
    /// Visible 3D object
    /// </summary>
    public interface IVisible : IPositionObject
    {
    }



       /// <summary>
    /// Consumer of visible 3D object
    /// </summary>
    public interface IVisibleConsumer
    {
        /// <summary>
        /// Adds visible object to consumer
        /// </summary>
        /// <param name="visible">Visible object to add</param>
        void Add(IVisible visible);

        /// <summary>
        /// Removes visible object from consumer
        /// </summary>
        /// <param name="visible">Visible object to remove</param>
        void Remove(IVisible visible);

        /// <summary>
        /// Post operation
        /// </summary>
        /// <param name="visible">Visible object</param>
        void Post(IVisible visible);
    }

Class VisibleConsumerLink links IVisibleConsumer to IVisible. Setting of VisibleConsumerLink arrow

   IVisible visible = ...;
            IVisibleConsumer consumer = ...;
            ICategoryArrow link = new VisibleConsumerLink();
            link.Source = consumer as ICategoryObject;
            link.Target = visible as ICategoryObject;   

implies execution of following operator

           consumer.Add(visible); 

Removing of VisibleConsumerLink follows that

        #region IRemovableObject Members

        /// <summary>
        /// Removing of VisibleConsumerLink
        /// </summary>
        void IRemovableObject.RemoveObject()
        {
                source.Remove(target);
        }

        #endregion

Following picture represents a sample of these interfaces.

 

Objects and arrows of above pictures have following types

N	Object name	Type	Implemented interfaces	Comment
1	Camera	WpfCamera	IVisibleConsumer	Wrapper of PerspectiveCamera
2	Globe	WpfShape	IVisible	Wrapper of Visual
3	Link	VisibleConsumerLink	ICategoryArrow	Links IVisibleConsumer with IVisible object

Above example means that Camera consumes Globe, and "consumes" means that visualizes.

5.2 Universality of architecture

This architecture operates with interfaces which do not know about WPF. So this software can be adapted to other 3D graphics technologies, for example OpenGL. I have already described compatibility of this software with different 3D technologies in "The time machine" article.

5.3 Interoperability with kinematics

Any object of 3D interface is associated with frame of reference. Next sample contains 3D object (Cube) with textures and four virtual cameras (Perspective Camera, X - camera, Y - camera and Z - camera).

 

The cube is linked by L 1 to moving frame of reference Frame 3 cameras are linked to fixed frames of reference. So cameras indicate Cube motion from different 3D points.

5.4 Features of WPF implementation of 3D graphics.

WPF implementation uses XAML file and texture files.

 

Following code is XAML file content

<ModelVisual3D xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation">
    <ModelVisual3D.Children>
        <ModelVisual3D>
            <ModelVisual3D.Content>
                <AmbientLight Color="#333333" />
            </ModelVisual3D.Content>
        </ModelVisual3D>
        <ModelVisual3D>
            <ModelVisual3D.Content>
                <DirectionalLight Color="#FFFFFF" Direction="-0.612372,-0.5,-0.612372" />
            </ModelVisual3D.Content>
        </ModelVisual3D>
        <ModelVisual3D>
            <ModelVisual3D.Content>
                <DirectionalLight Color="#FFFFFF" Direction="0.612372,-0.5,-0.612372" />
            </ModelVisual3D.Content>
        </ModelVisual3D>
        <ModelVisual3D>
            <ModelVisual3D.Content>
                <GeometryModel3D>
                    <GeometryModel3D.Material>
                        <MaterialGroup>
                            <DiffuseMaterial>
                                <DiffuseMaterial.Brush>
                                    <ImageBrush Stretch="UniformToFill" ImageSource="leaves_closeup.png" TileMode="None" ViewportUnits="Absolute" Viewport="0 0 1 1" AlignmentX="Left" AlignmentY="Top" Opacity="1.000000" />
                                </DiffuseMaterial.Brush>
                            </DiffuseMaterial>
                            <SpecularMaterial SpecularPower="85.3333">
                                <SpecularMaterial.Brush>
                                    <SolidColorBrush Color="#FFFFFF" Opacity="1.000000"/>
                                </SpecularMaterial.Brush>
                            </SpecularMaterial>
                        </MaterialGroup>

                    </GeometryModel3D.Material>
                    <GeometryModel3D.Geometry>
                        <MeshGeometry3D
      TriangleIndices="0,1,2 3,4,5"
      Normals="-1,0,0 -1,0,0 -1,0,0 -1,0,0 -1,0,0 -1,0,0 "
      TextureCoordinates="0,1 0,0 1,0 1,0 1,1 0,1 "
      Positions="-0.5,0.5,-0.5 -0.5,-0.5,-0.5 -0.5,-0.5,0.5 -0.5,-0.5,0.5 -0.5,0.5,0.5 -0.5,0.5,-0.5 " />
                    </GeometryModel3D.Geometry>
                </GeometryModel3D>
            </ModelVisual3D.Content>
        </ModelVisual3D>
        <ModelVisual3D>
            <ModelVisual3D.Content>
                <GeometryModel3D>
                    <GeometryModel3D.Material>
                        <MaterialGroup>
                            <DiffuseMaterial>
                                <DiffuseMaterial.Brush>
                                    <ImageBrush Stretch="UniformToFill" ImageSource="rocks.png" TileMode="None" ViewportUnits="Absolute" Viewport="0 0 1 1" AlignmentX="Left" AlignmentY="Top" Opacity="1.000000" />
                                </DiffuseMaterial.Brush>
                            </DiffuseMaterial>
                            <SpecularMaterial SpecularPower="85.3333">
                                <SpecularMaterial.Brush>
                                    <SolidColorBrush Color="#FFFFFF" Opacity="1.000000"/>
                                </SpecularMaterial.Brush>
                            </SpecularMaterial>
                        </MaterialGroup>

                    </GeometryModel3D.Material>
                    <GeometryModel3D.Geometry>
                        <MeshGeometry3D
      TriangleIndices="0,1,2 3,4,5"
      Normals="0,0,1 0,0,1 0,0,1 0,0,1 0,0,1 0,0,1 "
      TextureCoordinates="0,0 1,0 1,1 1,1 0,1 0,0 "
      Positions="-0.5,-0.5,0.5 0.5,-0.5,0.5 0.5,0.5,0.5 0.5,0.5,0.5 -0.5,0.5,0.5 -0.5,-0.5,0.5 " />
                    </GeometryModel3D.Geometry>
                </GeometryModel3D>
            </ModelVisual3D.Content>
        </ModelVisual3D>
        <ModelVisual3D>
            <ModelVisual3D.Content>
                <GeometryModel3D>
                    <GeometryModel3D.Material>
                        <MaterialGroup>
                            <DiffuseMaterial>
                                <DiffuseMaterial.Brush>
                                    <ImageBrush Stretch="UniformToFill" ImageSource= "branches.png" TileMode="None" ViewportUnits="Absolute" Viewport="0 0 1 1" AlignmentX="Left" AlignmentY="Top" Opacity="1.000000" />
                                </DiffuseMaterial.Brush>
                            </DiffuseMaterial>
                            <SpecularMaterial SpecularPower="85.3333">
                                <SpecularMaterial.Brush>
                                    <SolidColorBrush Color="#FFFFFF" Opacity="1.000000"/>
                                </SpecularMaterial.Brush>
                            </SpecularMaterial>
                        </MaterialGroup>

                    </GeometryModel3D.Material>
                    <GeometryModel3D.Geometry>
                        <MeshGeometry3D
      TriangleIndices="0,1,2 3,4,5"
      Normals="0,0,1 0,0,1 0,0,1 0,0,1 0,0,1 0,0,1 "
      TextureCoordinates="1,0 1,1 0,1 0,1 0,0 1,0 "
      Positions="0.5,-0.5,-0.5 0.5,0.5,-0.5 0.5,0.5,0.5 0.5,0.5,0.5 0.5,-0.5,0.5 0.5,-0.5,-0.5 " />
                    </GeometryModel3D.Geometry>
                </GeometryModel3D>
            </ModelVisual3D.Content>
        </ModelVisual3D>
        <ModelVisual3D>
            <ModelVisual3D.Content>
                <GeometryModel3D>
                    <GeometryModel3D.Material>
                        <MaterialGroup>
                            <DiffuseMaterial>
                                <DiffuseMaterial.Brush>
                                    <ImageBrush Stretch="UniformToFill" ImageSource="berries.jpg" TileMode="None" ViewportUnits="Absolute" Viewport="0 0 1 1" AlignmentX="Left" AlignmentY="Top" Opacity="1.000000" />
                                </DiffuseMaterial.Brush>
                            </DiffuseMaterial>
                            <SpecularMaterial SpecularPower="85.3333">
                                <SpecularMaterial.Brush>
                                    <SolidColorBrush Color="#FFFFFF" Opacity="1.000000"/>
                                </SpecularMaterial.Brush>
                            </SpecularMaterial>
                        </MaterialGroup>

                    </GeometryModel3D.Material>
                    <GeometryModel3D.Geometry>
                        <MeshGeometry3D
      TriangleIndices="0,1,2 3,4,5"
      Normals="1,0,0 1,0,0 1,0,0 1,0,0 1,0,0 1,0,0 "
      TextureCoordinates="1,0 1,1 0,1 0,1 0,0 1,0 "
      Positions="-0.5,-0.5,-0.5 -0.5,0.5,-0.5 0.5,0.5,-0.5 0.5,0.5,-0.5 0.5,-0.5,-0.5 -0.5,-0.5,-0.5 " />
                    </GeometryModel3D.Geometry>
                </GeometryModel3D>
            </ModelVisual3D.Content>
        </ModelVisual3D>
        <ModelVisual3D>
            <ModelVisual3D.Content>
                <GeometryModel3D>
                    <GeometryModel3D.Material>
                        <MaterialGroup>
                            <DiffuseMaterial>
                                <DiffuseMaterial.Brush>
                                    <ImageBrush Stretch="UniformToFill" ImageSource="Waterlilies.png" ViewportUnits="Absolute" 
            Viewport="0 0 1 1" AlignmentX="Left" AlignmentY="Top" Opacity="1.000000" />
                                </DiffuseMaterial.Brush>
                            </DiffuseMaterial>
                            <SpecularMaterial SpecularPower="85.3333">
                                <SpecularMaterial.Brush>
                                    <SolidColorBrush Color="#FFFFFF" Opacity="1.000000"/>
                                </SpecularMaterial.Brush>
                            </SpecularMaterial>
                        </MaterialGroup>

                    </GeometryModel3D.Material>
                    <GeometryModel3D.Geometry>
                        <MeshGeometry3D
      TriangleIndices="0,1,2 3,4,5 6,7,8 9,10,11"
      Normals="0,-1,0 0,-1,0 0,-1,0 0,-1,0 0,-1,0 0,-1,0 0,1,0 0,1,0 0,1,0 0,1,0 0,1,0 0,1,0 "
      TextureCoordinates="0,0 1,0 1,1 1,1 0,1 0,0 1,1 0,1 0,0 0,0 1,0 1,1 "
      Positions="-0.5,-0.5,-0.5 -0.5,0.5,0.5 -0.5,-0.5,0.5   0.5,-0.5,0.5 -0.5,-0.5,-0.5 -0.5,0.5,-0.5   
                          0.5,0.5,-0.5 -0.5,0.5,-0.5 -0.5,0.5,0.5  -0.5,0.5,0.5 0.5,0.5,0.5 0.5,0.5,-0.5 " />
                    </GeometryModel3D.Geometry>
                </GeometryModel3D>
            </ModelVisual3D.Content>
        </ModelVisual3D>
        <ModelVisual3D >
            <ModelVisual3D.Content>
                <GeometryModel3D>
                    <GeometryModel3D.Material>
                        <MaterialGroup>
                            <DiffuseMaterial>
                                <DiffuseMaterial.Brush>
                                    <ImageBrush Stretch="UniformToFill" ImageSource="Sunset.jpg" ViewportUnits="Absolute" 
            Viewport="0 0 1 1" AlignmentX="Left" AlignmentY="Top" Opacity="1.000000" />
                                </DiffuseMaterial.Brush>
                            </DiffuseMaterial>
                            <SpecularMaterial SpecularPower="85.3333">
                                <SpecularMaterial.Brush>
                                    <SolidColorBrush Color="#FFFFFF" Opacity="1.000000"/>
                                </SpecularMaterial.Brush>
                            </SpecularMaterial>
                        </MaterialGroup>

                    </GeometryModel3D.Material>
                    <GeometryModel3D.Geometry>
                        <MeshGeometry3D
      TriangleIndices="0,1,2 3,4,5 6,7,8 9,10,11"
      Normals="-1,0,0 -1,0,0 -1,0,0 -1,0,0 -1,0,0 -1,0,0 "
      TextureCoordinates="1,0 1,1 0,1 0,1 0,0 1,0 "
      Positions="-0.5,-0.5,0.5 -0.5,-0.5,-0.5 0.5,-0.5,-0.5   0.5,-0.5,-0.5 0.5,-0.5,0.5 -0.5,-0.5,0.5" />
                    </GeometryModel3D.Geometry>
                </GeometryModel3D>
            </ModelVisual3D.Content>
        </ModelVisual3D>
    </ModelVisual3D.Children>

    <ModelVisual3D.Transform>
        <Transform3DGroup >
            <Transform3DGroup.Children>
                <RotateTransform3D>
                    <RotateTransform3D.Rotation>
                        <AxisAngleRotation3D Angle="0" Axis="0 1 0" />
                    </RotateTransform3D.Rotation>
                </RotateTransform3D>
                <RotateTransform3D>
                    <RotateTransform3D.Rotation>
                        <AxisAngleRotation3D Angle="0" Axis="1 0 0" />
                    </RotateTransform3D.Rotation>
                </RotateTransform3D>
                <TranslateTransform3D  OffsetX="0" OffsetY="0" OffsetZ="0" />
            </Transform3DGroup.Children>
        </Transform3DGroup>
    </ModelVisual3D.Transform>
</ModelVisual3D>
    

This content has references to texture files, for example

<ImageBrush Stretch="UniformToFill" ImageSource="berries.jpg" TileMode="None" ViewportUnits="Absolute" Viewport="0 0 1 1" AlignmentX="Left" AlignmentY="Top" Opacity="1.000000" />
                                

WpfShape class is a wrapper of these objects and it serializes both XAML file and content of texture files.

             info.AddValue("Xaml", xaml);
            info.AddValue("Textures", textures, typeof(Dictionary<string, byte[]>));

Following code explains how textures dictionary is obtained.

             string[] files = new string[] // File names
            {
                "berries.jpg",
                "branches.png",
                "leaves_closeup.png",
                "rocks.png",
                "Sunset.jpg",
                "Waterlilies.png"
            };

            foreach (string fileName in files)
            {
                using (Stream stream = File.OpenRead(fileName))
                {
                    byte[] bytes = new byte[stream.Length];
                    stream.Read(bytes, 0, bytes.Length);     // Reads file
                    textures[fileName] = bytes;              // Fills texture dictionary
                }
            }     

6. Digital image processing + 3D graphics

6.1 Main idea

Interoperability between digital image processing and 3D graphics is supported by replacing texture images with digital image processing ones. Following code represents essential features interoperability class.

   /// <summary>
    /// 3D object with digital image processing interoperability
    /// </summary>
    [Serializable()]
    public class MotionImageFigure : WpfShape, IBitmapConsumer, IPostSetArrow
    {

        #region Fields

        /// <summary>
        /// Textures - Names of providers
        /// </summary>
        Dictionary<string, string> dTextures = new Dictionary<string, string>();

        /// <summary>
        /// Textures - Bitmap poviders
        /// </summary>
        Dictionary<string, IBitmapProvider> providers = new Dictionary<string, IBitmapProvider>();

        /// <summary>
        /// Post method
        /// </summary>
        protected virtual void Post()
        {
            foreach (string textureName in dTextures.Keys)         // Textures cycle
            {
                string providerName = dTextures[textureName];      // Name of provider
                if (providers.ContainsKey(providerName))
                {
                   IBitmapProvider p = providers[providerName];    // Bitmap provider
                   Bitmap bmp = p.Bitmap;                          // Bitmap of provider
                   paths.Remove(textureName);
                   using (MemoryStream stream = new MemoryStream())
                    {
                        bmp.Save(stream, System.Drawing.Imaging.ImageFormat.Bmp); // Saving bitmap to bypes
                        textures[textureName] = stream.GetBuffer();               // Replace texture by bitmap 
                    }
                }
            }
        }

Above Post() method yields replacement of texture images by digital image processing ones.

6.2 Elementary sample

Following picture represents usage of MotionImageFigure.

 

Type of Cube object is MotionImageFigure. This object has textures obtained from following graphical files:

• berries.jpg
• branches.png
• leaves_closeup.pn
• rocks.png
• Sunset.jpg
• Waterlilies.png

Some of these textures can be replaced as.

 

Here textures are replaced by following way.

N	Texture file	Name of provider for replacement
1	rocks.png	Compare
2	branches.png	Result of processing

• Download "Cube" sample 3.6 MB

6.3 Advanced sample

In my "Determination of Orbits of Artificial Satellites" article a complicated engineering task is considered. Recently this task was extended by 3D animation and audio (see "Theory versus Practice?"). Virtual artificial satellite yields following animation picture.

 

Following files are used for above 3D animation.

 

Now we would like replace earth.png texture by digital image processing result. In result we have following.

6.3.1 Installation of these samples

6.3.1.1 Installation of containes

Extract Containers.zip directory to directory of Aviation.exe file.

• Download Containers.zip 362.7 KB

4.2.7.4 Start animation

Start Aviation.exe.

Open OrbitImage.cfa or OrbitTwoImage.cfa.

• Download sample with processing of one image - 1.5 MB
• Download sample with processing of two images - 3.6 MB

Click following Animation button

Points of Interest

Writing of this article inspires recollections. When I was a young engineer I worked with very bad documents. Long time later I worked with documents of US companies. These documents were related to different areas. However the documents are clear since they had uniform style. After that I try to unify style of my own documents.