Click here to Skip to main content
Click here to Skip to main content

Tagged as

Go to top

XML mapping and validation in .NET

, 13 Jan 2014
Rate this:
Please Sign up or sign in to vote.
Carry information easily between xml with a different structure.

Introduction

Often I see big projects in which XML is used to convey information among different modules, (instead of JSON with a very good current speed to do it more efficiently) where there are several applications responsible to handle such information. The information handling is sometimes very "heavy", and some techniques a little bit dirty have to be used in order to replace lack of planning or design. Frameworks used can provide mechanism to settle those kind of errors, but sometimes the solution could be all but scalable, and sometimes even impossible to maintainable .

Context

In the case of XML, there are many solutions to solve most of the problems. One of them is the fact to get data from sources, which were not thought to be used where we want from, and due to it we have to develop a certain dirty code to get it working.

In this article, I will show how to map two different data sources, which structurally are equivalent, and data is get from one of them and it has to be put into the another one. Nonetheless they have been build of a different way and can’t be bound directly.

I'm going to show two solutions, using XPath and LINQ with C#.

We have two different data sources into XML, one for each. DataSource 1 and DataSource 2, that also have a different schema although it is equivalent. DataSource 2 is retrieved to pull out its information in order to fill DataSource 1 with it, we also want to remove or maintain (what it will depend on the situation) the current information of DataSource 1.

What do we need?

Imagine we obtain two XML structures which both contain an item collection of a same subjacent type (for example, cars, PCS, people, so on), so they just have a different namespace, more or less number of attributes as well as different names of themselves.

Just like this:

<?xml version="1.0" encoding="utf-8"?>
<DataSource1>
  <Items>
    <Item>
      <Name>Package 500-32</Name>
      <ID>32</ID>
      <Reference>Ref X003</Reference>
    </Item>
    <Item>
      <Name>Package 501-54</Name>
      <ID>54</ID>
      <Reference>Ref X003</Reference>
    </Item>
  </Items>
</DataSource1> 
<?xml version="1.0" encoding="utf-8"?>
<DataSource2>
  <ItemCollection>
    <ItemInformation>
      <Package>Package 500-35</Package>
      <Reference>Ref X003</Reference>
      <IDPACK>35</IDPACK>
      <OtherField>info</OtherField>
    </ItemInformation>
</ItemCollection>
</DataSource2>

When we have already obtained the information, in our case it may mean that the current information after a refresh has only to have one element. Furthermore it was retrieved with other format, so it has to be mapped into the corresponding data source with the correct format.

The data structure really is the same, but certain names and possible attributes don’t match, and we cannot use ReplaceSelf() XPath method as it would throw an exception, due to the different definition of the XSD scheme of the sources.

As it´s not the same, we have to provide a mechanism to specify how the links have to be related.

Using the code

Some applications have to do this many times, thus if there is not an appropriate framework or functionality to get a suitable handling, as it is said, it can result to something dirty or tedious.

We just have to specify the data sources references, the node content to be put into end data source and the node naming map between sources.

Given DataSource1 and DataSource2 in the example above, take a look at the next sentence:

Map(true, DataSource2, "//ItemInformation", DataSource1, "//Item",  
new Dictionary<string, string>
                    {
                        { "Package", "Name" },
                        { "IDPACK", "ID" },
                        { "Reference", "Reference" },
                    }); 

With this simple sentence, we get the work done. The first parameter is used to empty or not the xml, depending if the information has to be all replaced or concatenated.

The implementation using XPath is the next:

public void Map(bool empty, XPathNavigator xmlSource, string nodeFiller, 
            XPathNavigator xmlToFill, string nodeFilled,
            Dictionary<string, string> MapNodes)
{
    if(empty) Empty(xmlToFill, nodeFilled, 1);
    XPathNodeIterator
        iteratorXmlToFill = xmlToFill.Select(nodeFilled),
        iteratorXmlSource = xmlToFill.Select(nodeFilled);
    while (iteratorXmlToFill.MoveNext()) ;
    XPathNavigator CloneReference = iteratorXmlToFill.Current.Clone();
    while (iteratorXmlSource.MoveNext())
    {
        if (iteratorXmlSource.CurrentPosition > 1)
        {
            iteratorXmlToFill.Current.InsertAfter(CloneReference);
            iteratorXmlToFill = xmlToFill.Select(nodeFilled);
            while (iteratorXmlToFill.MoveNext()) ;
        }
        XPathNodeIterator subiterator = 
          iteratorXmlSource.Current.SelectChildren(XPathNodeType.Element);
        while (subiterator.MoveNext())
        {
            string valor = subiterator.Current.Name;
            if (MapNodes.ContainsKey(valor))
            {
                iteratorXmlToFill.Current.SelectSingleNode(
                  MapNodes[valor]).SetValue(subiterator.Current.Value);
            }
        }
    }
}

// Empty a XML Structure taking care of the xsd schema validation minimum occurrences.
public void Empty(XPathNavigator xml, string node, int MinOccur)
{
    XPathNodeIterator iterator = xml.Select(node);
    while (iterator.Count >= minimunNumberOcurrences)
    {
        iterator = xml.Select(node);
        if (iterator.MoveNext() && iterator.Count > MinOccur)
        {
            iterator.Current.DeleteSelf();
        }
        else
        {
            XPathNodeIterator subiterator = iterator.Current.SelectChildren(XPathNodeType.Element);
            while (subiterator.MoveNext())
            {
                subiterator.Current.SetValue(string.Empty);
            }
            break;
        }
    } 
}

* The double slash in the node parameters is due to reference the node bunch because they are supposed to be repeated nodes, and the method is using XPathNavigator.Select() to get the collection which works properly with that XPath specification.

** The dictionary is a handy way to map value pairs, but indeed there are many more alternatives.

Ok, but where is the validation, or even a filter?

We should be able to decide what nodes have to put into our end source as well as specify what conditions they have to fulfill.

Of course we can use the XPath filters written in the node parameters themselves, like "//Item[IDPACK<5]" would return the items which have an IDPACK lesser than five, and then they'd be put into the specified source. But what's the matter if we want going beyond and be able to do any kind of action?. I might require do a group of actions for each element that it has to be parsed and introduced in the final data source.

We can send a pointer function, I mean, a delegate or a delegates array to make those kind of stuff, to provide a customized group of actions each time.

On the beaming process, it exists an iteration over each item parsed, the validation makes sense if we analyze these items.

In order to achieve this fact, we overload the function.

For example, with the next statement we’d filter those items which IDPACK were lesser than five and not contain the text "bad state" in their subnode Package.

Map(true, DataSource2, "//ItemInformation", DataSource1, "//Item",  
    new Dictionary<string, string>
                    {
                        { "Package", "Name" },
                        { "IDPACK", "ID" },
                        { "Reference", "Reference" },
                    },
    new ValidationFunction[] {
                     new ValidationFunction((value1,value2) => value1.Equals(value2)),
                           myOwnValidationFunction
                },
                new string[] {
                    "IDPACK",
                    "Package"
                },
                new string[] {
                    "5",
                    "bad state"
                } 
             );
public bool myOwnValidationFunction(string v1, string v2)
{
    return !v1.Contains(v2);
}

And we have just had to create our validation functions. In the previous example we use two of them, one already declared and the other anonymously in the sentence itself.

The process is the next:

public delegate bool ValidationFunction(string value1, string value2);

public void Map(bool empty, XPathNavigator xmlSource, string nodeFiller,
                XPathNavigator xmlToFill, string nodeFilled,
                Dictionary<string, string> MapNodesValues,
                ValidationFunction[] validationFunctions,
                string[] validationSubNodes, string[] validationValues)
        {
            if (empty) Empty(xmlToFill, nodeFilled, 1);

            XPathNodeIterator
                iteratorXmlToFill = xmlToFill.Select(nodeFilled),
                iteratorXmlSource = xmlSource.Select(nodeFiller);

            while (iteratorXmlToFill.MoveNext()) ;

            XPathNavigator CloneReference = iteratorXmlToFill.Current.Clone();

            while (iteratorXmlSource.MoveNext())
            {
                var validate = true;
                int i = 0;
                while ((i < validationFunctions.Length) && validate)
                {
                    if (!validationFunctions[i](iteratorXmlSource.Current.SelectSingleNode(validationSubNodes[i]).Value, validationValues[i]))
                    {
                        validate = false;
                    }
                    i++;
                }
                if (validate)
                {
                    if (iteratorXmlToFill.CurrentPosition > 1)
                    {
                        iteratorXmlToFill.Current.InsertAfter(CloneReference);
                        iteratorXmlToFill = xmlToFill.Select(nodeFilled);

                        while (iteratorXmlToFill.MoveNext()) ;
                    }

                    XPathNodeIterator subiterator = iteratorXmlSource.Current.SelectChildren(XPathNodeType.Element);

                    while (subiterator.MoveNext())
                    {
                        string valor = subiterator.Current.Name;

                        if (MapNodesValues.ContainsKey(valor))
                        {
                            iteratorXmlToFill.Current.SelectSingleNode(MapNodesValues[valor]).SetValue(subiterator.Current.Value);
                        }
                    }
                }

            }
        } 

Combining XPath filtering with this customized mechanism of filter and validation we should be able to do any transfer from one data source to another one with XPath.

This whole procedure can be useful, as I said, to bind some points with different format.

At last, I show how we can realize the process in an better way using Linq:

public static void Map(bool empty, XPathNavigator xmlSource, string nodeFiller,
                XPathNavigator xmlToFill, string nodeFilled,
                Dictionary<string, string> MapNodesValues,
                ValidationFunction[] validationFunctions,
                string[] validationSubNodes, 
        string[] validationValues, 
        int minimumOcurrencesFinalSource)
        {
            XPathNavigator xmlNodeFinalModel = xmlToFill.SelectSingleNode(nodeFilled);

            XPathNavigator xmlNodeFinalUsage = xmlNodeFinalModel.Clone();

            if (empty) Empty(xmlToFill, nodeFilled, minimumOcurrencesFinalSource);

            var SourceNodes =
                from XPathNavigator sourceNode in (xmlSource.Select(nodeFiller))
                where Validation(sourceNode, 
                validationFunctions, 
                validationSubNodes, 
                validationValues)
                select sourceNode;

            XPathNodeIterator SubXmlToFillIterator = xmlToFill.Select(nodeFilled);
            SubXmlToFillIterator.MoveNext();

            int initial = minimumOcurrencesFinalSource;
            while (initial <= minimumOcurrencesFinalSource)
            {
                SubXmlToFillIterator.MoveNext();
                initial++;
            }
            XPathNavigator SubXmlToFill = SubXmlToFillIterator.Current;

            foreach (var sourceNode in SourceNodes)
            {
                xmlNodeFinalUsage = xmlNodeFinalModel.Clone();
                XPathNodeIterator subiterator = 
            sourceNode.SelectChildren(XPathNodeType.Element);
                string name = string.Empty;
                while (subiterator.MoveNext())
                {
                    name = MapNodesValues[subiterator.Current.Name];
                    xmlNodeFinalUsage.SelectSingleNode(name).SetValue(
            subiterator.Current.Value);
                }
                if (!SourceNodes.LastOrDefault().OuterXml.Equals(sourceNode.OuterXml))
                {
                    SubXmlToFill.InsertBefore(xmlNodeFinalUsage);
                    SubXmlToFill.MoveToNext();
                }
            }
                   
        }

public static bool Validation(XPathNavigator xmlNode, 
            ValidationFunction[] validationFunctions, 
            string[] validationSubNodes, 
            string[] validationValues)
        {
            bool validate = true;
            int i = 0;
            while (validate && (i < validationFunctions.Length))
            {
                validate = validationFunctions[i](
                        xmlNode.SelectSingleNode(
                    validationSubNodes[i]).Value, 
                                 validationValues[i]);
                i++;
            }
            return validate;
} 
     
//In the next example, I show the process of retrieving data from a database, mapping between two different XML Data Sources and writing it into a xml file. The data details maintain the same names and fields used in the previous examples. 

//We receive a reference to end data source

public void Example(XPathNavigator DataSource1)
{
    string mydataconnection = "...";
    using (SqlConnection con = new SqlConnection(mydataconnection))
    {
        DataSet ds = new DataSet();
        string myTable = "Table_itemcollection";

        // Get data from the database
        SqlDataAdapter da =
            new SqlDataAdapter(
        "select Package,Reference,IDPACK,OtherField from " + myTable, con);
        da.Fill(ds, myTable);
        XPathNavigator DataSource2 = new XmlDataDocument(ds).CreateNavigator();

        // Map xml information between xml sources
        Map(true, DataSource2, "//Item", DataSource1, "//ItemInformation",
            new Dictionary<string, string>
            {
                { "Package", "Name" },
                { "IDPACK", "ID" },
                { "Reference", "Reference" },
            });

        // Write the results
        XmlDocument doc = new XmlDocument();
        doc.LoadXml(DataSource1.OuterXml);
        doc.Save("myfile.xml");
    }
}  

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Share

About the Author

Juan Carlos Recio Abad
Software Developer
Spain Spain
I´m a Computer Engineer from Spain with a lot of interest and passion about the world of technologies, I love all kind of them, as well as I do it for the science.
I like to create many sorts of utilities and tools, inventing solutions and methods to get solving the challenges which can appear.
 
I also love math and Physics, its misteries and incredible solutions, so I've always been automatizing algorithms and processes related to them on my own, whether for fun or another affairs.
 
Of course, I enjoy reading history, literature, music, and play sport!

Comments and Discussions

 
QuestionMy Vote of 5 PinmemberRubol9-Dec-13 18:18 
QuestionNeeds formatting PinmvpRichard MacCutchan6-Dec-13 2:37 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

| Advertise | Privacy | Mobile
Web04 | 2.8.140905.1 | Last Updated 13 Jan 2014
Article Copyright 2013 by Juan Carlos Recio Abad
Everything else Copyright © CodeProject, 1999-2014
Terms of Service
Layout: fixed | fluid