Click here to Skip to main content
Click here to Skip to main content

Design Pattern for Overriding XML

By , 28 Oct 2008
 

Introduction

With XML being widely used for defining various system configurations, there is sometimes a need to be able to define some base configuration and then override certain XML elements in separate configuration files for each specific context.

The following article presents a generic design pattern for overriding XML elements, and also provides a generic XSL stylesheet based on this pattern that allows merging an overriding XML file with the corresponding base XML structure.

XML Override Design Pattern

The idea behind this design pattern is for the overriding XML to repeat the structure of the base XML to the extent necessary to override the base values and to specify new values to be used. The schema for the overriding XML can be almost the same as for the base XML, except that each element would be not required to enable specifying just a part of the XML tree that is needed for overriding. In addition, each element can have an optional extra attribute overrideMode to specify how the element overrides the corresponding base element. This attribute can assume one of the following three values:

  • delete – this value can be used on any element to indicate that this element including all its child elements should be deleted from the XML tree.
  • update – this value can be used on any non-leaf override element to indicate that the current base element must be updated with the override element, but all child elements should not be changed unless specifically overridden further in the XML. This is driven by the fact that non-leaf elements in the overriding XML could serve a dual purpose: to provide new values for that element, and to provide the structure for its child override elements. Override XML elements that have no children require no extra attribute at all since their very presence would automatically indicate either overriding of the existing elements or adding new elements, respectively.
  • replace – this value can be used on non-leaf elements to completely replace an element including all its child elements.

With this additional attribute, it is possible to easily override any part of the base XML in a pretty straightforward manner. To demonstrate how it works, let us consider a sample base XML that consists of a mix of first and second level elements, attributes, and text nodes, as follows:

<root>
    <lvl1 id="elem11" value="val11">
        <lvl2 name="elem21" value="val21">txt21</lvl2>
        <lvl2 name="elem22" value="val22">txt22</lvl2>txt11</lvl1>
    <lvl1 id="elem12" value="val12">
        <lvl2 name="elem21" value="val21">txt21</lvl2>
        <lvl2 name="elem22" value="val22">txt22</lvl2>txt12</lvl1>
    <lvl1 id="elem13" value="val13">
        <lvl2 name="elem21" value="val21">txt21</lvl2>
        <lvl2 name="elem22" value="val22">txt22</lvl2>txt13</lvl1>
    <lvl1 id="elem14" value="val14">
        <lvl2 name="elem21" value="val21">txt21</lvl2>
        <lvl2 name="elem22" value="val22">txt22</lvl2>txt14</lvl1>
    <lvl1 id="elem15" value="val15">
        <lvl2 name="elem21" value="val21">txt21</lvl2>
        <lvl2 name="elem22" value="val22">txt22</lvl2>txt15</lvl1>
</root>

Now, the following sample override XML shows some examples of how to override different base elements in different ways. The comments in the XML explain how each element is being overridden.

<root>
    <!-- update elem11 and the two child elements respectively -->
    <lvl1 id="elem11" value="oval11" overrideMode="update">
        <lvl2 name="elem21" value="oval21">otxt21</lvl2>
        <lvl2 name="elem22" value="oval22">otxt22</lvl2>otxt1</lvl1>
    <!-- update elem12, since it has no children -->
    <lvl1 id="elem12" value="oval12">otxt2</lvl1>
    <!-- update children of elem13 but not the element itself, 
         since the overrideMode is not specified -->
    <lvl1 id="elem13" value="oval13(ignored)">
        <lvl2 name="elem21" value="oval21"/>
        <lvl2 name="elem22" value="oval22"/>
        <lvl2 name="elem23" value="oval23"/>otxt3</lvl1>
    <!-- delete elem14 -->
    <lvl1 id="elem14" overrideMode="delete"/>
    <!-- replace elem15 with the following element -->
    <lvl1 id="elem15" value="oval12" overrideMode="replace">
        <lvl2 name="elem21" value="oval21"/>otxt5</lvl1>
</root>

Generic XSLT Implementation

One great thing about this design pattern is that it allows creating a generic transformation template that would work for most of the cases. It uses an XSLT template named merge that can be used either by other templates as part of the transformation, or in a standalone XSL to apply the overriding XML to the base XML.

The only limitation this XSLT imposes is that each element would either be one of a kind, or have its first attribute as a key that could uniquely identify the element within its parent element. This is a pretty reasonable limitation that should cover most of the cases, since in order to allow overriding XML elements, they need to have a key, unless they are a single child of their parent of its kind, and the first element seems to be a good choice to define the key. Below is the generic XSLT template that implements this pattern.

<?xml version="1.0" encoding="UTF-8"?>
<xsl:stylesheet version="2.0" 
     xmlns:xsl="http://www.w3.org/1999/XSL/Transform" 
     xmlns:xs="http://www.w3.org/2001/XMLSchema">
  <xsl:output indent="yes"/>
  <xsl:param name="overrideFile"/>
  <xsl:template match="/">
    <xsl:call-template name="merge">
      <xsl:with-param name="std" select="."/>
      <xsl:with-param name="ovrd" select="document($overrideFile)"/>
    </xsl:call-template>
  </xsl:template>
  <xsl:template name="merge">
    <xsl:param name="std"/>
    <xsl:param name="ovrd"/>
    <xsl:for-each select="$std/*">
      <xsl:variable name="key" select="@*[1]"/>
      <xsl:variable name="ovr" 
         select="$ovrd/*[local-name() = local-name(current()) 
                 and (not($key) or @*[1] = $key)]"/>
      <xsl:if test="count($ovr) = 0">
        <xsl:copy-of select="."/>
      </xsl:if>
      <xsl:if test="count($ovr) = 1 and (not($ovr/@overrideMode) 
                    or $ovr/@overrideMode != 'delete')">
        <xsl:choose>
          <xsl:when test="count($ovr/*) = 0 or $ovr/@overrideMode = 'update' 
                          or $ovr/@overrideMode = 'replace'">
            <xsl:variable name="current" select="."/>
            <xsl:for-each select="$ovr">
              <xsl:copy>
                <xsl:for-each select="@*[name() != 'overrideMode']|
                                      text()[string-length(normalize-space(.))>0]">
                  <xsl:copy/>
                </xsl:for-each>
                <xsl:choose>
                  <xsl:when test="$ovr/@overrideMode = 'replace'">
                    <xsl:copy-of select="*"/>
                  </xsl:when>
                  <xsl:otherwise>
                    <xsl:call-template name="merge">
                      <xsl:with-param name="std" select="$current"/>
                      <xsl:with-param name="ovrd" select="."/>
                    </xsl:call-template>
                  </xsl:otherwise>
                </xsl:choose>
              </xsl:copy>
            </xsl:for-each>
          </xsl:when>
          <xsl:otherwise>
            <xsl:copy>
              <xsl:for-each select="@*|text()[string-length(normalize-space(.))>0]">
                <xsl:copy/>
              </xsl:for-each>
              <xsl:call-template name="merge">
                <xsl:with-param name="std" select="."/>
                <xsl:with-param name="ovrd" select="$ovr"/>
              </xsl:call-template>
            </xsl:copy>
          </xsl:otherwise>
        </xsl:choose>
      </xsl:if>
    </xsl:for-each>
    <xsl:for-each select="$ovrd/*">
      <xsl:variable name="key" select="@*[1]"/>
      <xsl:if test="count($std/*[local-name() = local-name(current()) 
                    and (not($key) or @*[1] = $key)]) = 0">
        <xsl:copy-of select="."/>
      </xsl:if>
    </xsl:for-each>
  </xsl:template>
</xsl:stylesheet>

Conclusion

This article illustrates how a common pattern of defining overriding XML can be achieved by using pretty much the same structure as the original XML plus an additional attribute for each element. This pattern can leverage a generic XSLT implementation that can apply the overriding XML to the base XML and transform it into the resulting XML.

History

  • 10/28/2008: The first version of this article has been posted.

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

About the Author

Serghei Sarafudinov
President SIS Consulting, LLC
United States United States
Member
President and CEO of SIS Consulting, LLC – an offshore software development company.
Also a Senior Consultant at Princeton Consultants, Inc. – a US-based IT and Management consulting company.

Sign Up to vote   Poor Excellent
Add a reason or comment to your vote: x
Votes of 3 or less require a comment

Comments and Discussions

 
You must Sign In to use this message board.
Search this forum  
    Spacing  Noise  Layout  Per page   
GeneralAttribute order is implementation-dependentmemberses4j7 Jul '10 - 9:24 
Hi -
 
The XSLT specification[^] unfortunately seems to say that attribute ordering is implementation-dependent, and so the concept and effect of "first attribute as key" may vary by XSLT engine. Too bad, but making the solution more standards-compliant probably requires adding some kind of overrideModeKey attribute to use for matching (that defaults to "id" maybe.)
 
Scott
GeneralRe: Attribute order is implementation-dependentmemberSerghei Sarafudinov7 Jul '10 - 9:32 
That's correct. However in practicality most (if not all) XSLT engines are using the natural order of the attributes in which they are listed.
If the engine you are using is doing it differently or if you want to make it more robust or if your key is not the first attribute, then you can definitely add a new attribute that designates the key attribute.
 
Good point though.
 
Serghei
QuestionAttribute merge capability?memberses4j7 Jul '10 - 7:50 
Hi -
 
Do you have any ideas about how to extend this design pattern (and xslt implmentation) to include attribute merge semantics? It would be useful to me to be able to have an override file describe that the attributes should be merged, such that
 
base:
 
 <MyNode attr1="orig1" attr2="orig2" />
 
override:
 
 <MyNode attr1="new1" attr3="new3" />
 
would output:
 <MyNode attr1="new1" attr2="orig2" attr3="new3" />
 
Maybe an additional overrideMode? Or a separate one that controls attribute handling?
AnswerRe: Attribute merge capability?memberSerghei Sarafudinov7 Jul '10 - 8:17 
Hi, Scott.
 
Thanks for your input. Originally my thinking was that if you don't have many attributes, then it's no big deal to just use the 'update' mode and specify all attributes in the override node. This way you can also delete attributes if needed. But if you have a lot of attributes on your nodes, then I can see a point in supporting merging attributes.
 
In this case I think the best way to enhance it would be to add another override mode (e.g. 'merge' or something). You will need to update the stylesheet then to copy only the current attributes that are not listed in the override node and then copy the attributes from the override node (just as in the update mode).
 
I don't have much time for testing it now, but if you can post a solution that works for you, that would be great.
 
Best regards,
Serghei
GeneralRe: Attribute merge capability?memberses4j7 Jul '10 - 9:18 
I'll try, but I'm pretty bad at XSLT. If you can whip off a "here's the concept" snippet of xslt, I'm happy to test/fix it... Wink | ;)
Generalbugfix for nodes with only an overrideMode attributememberses4j7 Jul '10 - 7:36 
Hi -
 
The script seems to have a small bug when you have a node with overrideMode specified and no other key. Without this patch, it dumps your new node into the output twice, once in the proper spot, and again at the end with the overrideMode tag in place.
 
Replace:
    <xsl:for-each select="$ovrd/*">
      <xsl:variable name="key" select="@*[1]"/>
      <xsl:if test="count($std/*[local-name() = local-name(current()) 
                    and (not($key) or @*[1] = $key)]) = 0">
 
with this slightly modified code:
 
      
    <xsl:for-each select="$ovrd/*">
      <xsl:variable name="key" select="@*[not(name() = 'overrideMode')][1]"/>
      <xsl:if test="count($std/*[local-name() = local-name(current()) 
                    and (not($key) or @*[1] = $key)]) = 0">
 
Scott Stafford
GeneralRe: bugfix for nodes with only an overrideMode attributememberSerghei Sarafudinov7 Jul '10 - 12:07 
Could you post an example to show what kind of setup caused such behavior?
GeneralMultiple/Recursive Files [modified]memberSoapSeller13 Apr '10 - 4:22 
I've made a version that replace the overrideFile file parameter with tags for base files.
The xslt is recursive(can take Include within include).
 
Include syntax is <Include>BaseFile.xml</include>.
 
XSLT:
<?xml version="1.0" encoding="UTF-8"?>
<!-- Original Source: http://www.codeproject.com/KB/XML/XMLOverride.aspx -->
<xsl:stylesheet version="2.0"
     xmlns:exslt="http://exslt.org/common"
     xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
     xmlns:xs="http://www.w3.org/2001/XMLSchema">
  <xsl:output indent="yes"/>
  
  <!-- Start of everything -->
  <xsl:template match="/">
      <xsl:call-template name="start_merge">
        <xsl:with-param name="current" select="."/>
      </xsl:call-template>
  </xsl:template>
  
  <!-- get a document, merge the includes, and override them with the document -->
  <xsl:template name="start_merge">
    <xsl:param name="current"/>
    <xsl:choose>
      <xsl:when test="count($current/Parameters/Include) > 0">
        <xsl:variable name="incs">
          <xsl:call-template name="merge_incs">
            <xsl:with-param name="current" select="document($current/Parameters/Include[1]/text())"/>
            <xsl:with-param name="all" select="$current/Parameters/Include[position()>1]/text()"/>
          </xsl:call-template>
        </xsl:variable>
        <xsl:call-template name="merge">
          <xsl:with-param name="std" select="exslt:node-set($incs)"/>
          <xsl:with-param name="ovrd" select="$current"/>
        </xsl:call-template>
      </xsl:when>
      <xsl:otherwise>
        <xsl:copy-of select="$current"/>
      </xsl:otherwise>
    </xsl:choose>
  </xsl:template>
  
  <!-- Get a list of includes, and override them with each other(last override the first) -->
  <xsl:template name="merge_incs">
    <xsl:param name="current"/>
    <xsl:param name="all"/>
 
    <xsl:variable name="current_merged">
      <xsl:call-template name="start_merge">
        <xsl:with-param name="current" select="$current"/>
      </xsl:call-template>
    </xsl:variable>
    <xsl:choose>
      <xsl:when test="count($all) > 0">
          <xsl:variable name="all_merged">
          <xsl:call-template name="merge_incs">
            <xsl:with-param name="current" select="document($all[1])"/>
            <xsl:with-param name="all" select="$all[position()>1]"/>
          </xsl:call-template>
        </xsl:variable>
 
        <xsl:call-template name="merge">
          <xsl:with-param name="std" select="exslt:node-set($current_merged)"/>
          <xsl:with-param name="ovrd" select="exslt:node-set($all_merged)"/>
        </xsl:call-template>
      </xsl:when>
      <xsl:otherwise>
        <xsl:copy-of select="$current_merged"/>
      </xsl:otherwise>
    </xsl:choose>
  </xsl:template>
 
  <!-- Ovrerride a document(std) with another(ovrd). Ignore <Include> tags -->
  <xsl:template name="merge">
    <xsl:param name="std"/>
    <xsl:param name="ovrd"/>
    <xsl:for-each select="$std/*">
      <xsl:if test="not(local-name() = 'Include')">
        <xsl:variable name="key" select="@*[1]"/>
        <xsl:variable name="ovr"
           select="$ovrd/*[local-name() = local-name(current()) 
                 and (not($key) or @*[1] = $key)]"/>
        <xsl:if test="count($ovr) = 0">
          <xsl:copy-of select="."/>
        </xsl:if>
        <xsl:if test="count($ovr) = 1 and (not($ovr/@overrideMode) 
                    or $ovr/@overrideMode != 'delete')">
          <xsl:choose>
            <xsl:when test="count($ovr/*) = 0 or $ovr/@overrideMode = 'update' 
                          or $ovr/@overrideMode = 'replace'">
              <xsl:variable name="current" select="."/>
              <xsl:for-each select="$ovr">
                <xsl:copy>
                  <xsl:for-each select="@*[name() != 'overrideMode']|
                                      text()[string-length(normalize-space(.))>0]">
                    <xsl:copy/>
                  </xsl:for-each>
                  <xsl:choose>
                    <xsl:when test="$ovr/@overrideMode = 'replace'">
                      <xsl:copy-of select="*"/>
                    </xsl:when>
                    <xsl:otherwise>
                      <xsl:call-template name="merge">
                        <xsl:with-param name="std" select="$current"/>
                        <xsl:with-param name="ovrd" select="."/>
                      </xsl:call-template>
                    </xsl:otherwise>
                  </xsl:choose>
                </xsl:copy>
              </xsl:for-each>
            </xsl:when>
            <xsl:otherwise>
              <xsl:copy>
                <xsl:for-each select="@*|text()[string-length(normalize-space(.))>0]">
                  <xsl:copy/>
                </xsl:for-each>
                <xsl:call-template name="merge">
                  <xsl:with-param name="std" select="."/>
                  <xsl:with-param name="ovrd" select="$ovr"/>
                </xsl:call-template>
              </xsl:copy>
            </xsl:otherwise>
          </xsl:choose>
        </xsl:if>
      </xsl:if>
    </xsl:for-each>
    <xsl:for-each select="$ovrd/*">
      <xsl:variable name="key" select="@*[1]"/>
      <xsl:if test="count($std/*[local-name() = local-name(current()) 
                    and (not($key) or @*[1] = $key)]) = 0">
        <xsl:copy-of select="."/>
      </xsl:if>
    </xsl:for-each>
  </xsl:template>
</xsl:stylesheet>
modified on Tuesday, April 13, 2010 4:36 PM

GeneralExactly what I was looking for!membermannabaron7 Feb '09 - 3:00 
Hi Serghei,
thank you very much for saving me the time realizing what you developed.
 
I need this also for merging configuration files.
I have default configuration files containing the standard settings and
custom configuration files containing the special settings.
Merging a default with a custom configuration file will lead to the
resulting configuration file used in the application.
 
A possible enhancement would be the possibility to sort some elements.
But of course this could be realized in an own XSL file.
 
Anyway, thank you very much!
Greetings, Wolfgang Mena-Bruhn
AnswerRe: Exactly what I was looking for!memberSerghei Sarafudinov7 Feb '09 - 9:24 
Thanks for your feedback, I am glad you found it helpful.
 
Yes, one limitation of this implementation is an ability to easily insert new elements at a certain position rather than just adding them at the end. Typically, with relatively small lists this can be worked around by using the replace override mode to override the whole list in the right order. But if the list is quite large or has child elements then I agree that this can get pretty cumbersome and a better option would be to enhance this implementation.
 
Serghei
GeneralAhh, I seememberDmitri Nesteruk2 Nov '08 - 20:11 
All right, so you seem to be applying OOP principles to XML instead of classes. I can see how this can be useful in some situations. One I commonly encounter is where XML schemas from different parties differ somehow and you have to remap one to another, adding/removing elements in order to restore the complete picture. But I don't think that what you are describing here can be made automatically - i.e., XSD.exe probably has no clue about XML inheritance. Smile | :)
GeneralRe: Ahh, I seememberSerghei Sarafudinov4 Nov '08 - 17:12 
Well, the Override pattern is not so much related to the OOP principles as to the idea of trying to keep every piece of information in no more than one place. A typical application of this pattern could be an XML configuration file, which you can partially override for a specific context. For example, ASP.NET supports a similar concept with the web.config files where you can use <add .../>, <remove .../> and <clear .../> tags to override the base config file.
 
Obviously, another generic way to override any XML file would be to write an XSLT transformation that adds, removes or replaces elements in the base XML. This is an ultimately flexible way to override XML files, but is far more complicated than the one proposed in the current article.
 
This pattern can probably be also applied to XSD files as long as the elements are identifiable within their parent elements either by a key attribute or by the kind of the element.
 
As you said, this would not happen automatically - you'd need to run the base XML through an XSLT that would apply the override file to it before using the result further, e.g. running XSD.exe against it.

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Rant Rant    Admin Admin   

Permalink | Advertise | Privacy | Mobile
Web02 | 2.6.130523.1 | Last Updated 28 Oct 2008
Article Copyright 2008 by Serghei Sarafudinov
Everything else Copyright © CodeProject, 1999-2013
Terms of Use
Layout: fixed | fluid