In a nutshell: Microsoft's Domain Specific Software Factory Examples

Introduction

After looking at the direction Microsoft and Jack Greenfield and his team are going with Software Factories and the big international push in OOPSLA 2005 last year towards this methodology, I got interested in Microsoft's motivation for this new methodology. The methodology basically states that to allow generalities for domain specific code to occur, a template or basic design can be supplied to a local designer. I realize that is an over simplistic description which leaves out a lot of important details, but it is as complex as I dare to go right now. MS's Domain Specific Language examples, if you care to look at them, are mostly geared to allowing code, use cases, and general workflow to be templated, or pre-modeled for a particular group of domain specific functions.

This article is partially based on the Microsoft Methodology surrounding Software Factories, and partially on some thoughts around template driven design in general.

What is a Software Factory?

If you look directly at the Microsoft Domain Specific Language examples, you will see what they are introducing is a way to build specific types of projects, with all the UI, class definitions, workflow definitions.. etc., set up before the developer actually starts the code. So, let's say, an architect for a company decides that the company needs a generalized way to build a particular process flow to solve a particular business problem. He can set up a project that defines the overall problem, and ship that 'template' out to his development teams, which actually builds code based on that template project. This works kind of like project definitions in .NET do now, except instead of having a Web Forms, WinForms, Web Service, or Console project, an architectural designer can put together a series of definitions based on use cases, process workflow, or class definitions, and create a whole new specialized type of project for a particular business problem.

Let's look at a term that is being used in regard to this methodology: Domain Model. A Domain Model, in a nutshell, is a description or sets of rules for describing a particular process flow grouping. It is a way to catalog a particular organization set. In the DSL examples, it described the project definition for the output project. The software factory is, in effect, the domain model. This model allows the outskirts of a particular domain to be defined. If we wanted to define a process flow for building a series of relationships between classes, and group that process flow, we do that in the domain model.

Let's walk through a simple scenario and see how this might work. Your architect has determined that developers are building Widget A for every project, and he wants to make Widget A a standardized template. So the architect begins by defining what Widget A can accomplish overall. The first thing he wants to define is a top level object called PageFlow. He will define it in the correct namespace with all the class attributes he needs:

<concepts>
    <concept name="PageFlow" 
       identity="17c43f6b-9bdc-4f78-a245-13c7b8357e5a" 
       namespace="Fabrikam.UIPChart.DomainModel" 
       id="i17c43f6bz9bdcz4f78za245z13c7b8357e5a" 
       isLoaded="true" isAbstract="false">
           <mdfmetadata xsi:type="conceptorshapemdfmetadata" 
                 accessmodifier="public" category="" 
                 description="" doccomment="" localize="false" />
    </concept>

Next, he wants to define a child class that will have a relationship to the parent PageFlow object. This class will be the actual class later that has action within the output project. PageFlow will be like a container object, but Pages will be a collection set in this example. This object also has attributes, accessors, and types defined:

<concept name="Page" 
   identity="bc358cb4-8423-406b-beef-8f030a7ceea6" 
   namespace="Fabrikam.UIPChart.DomainModel"
   id="ibc358cb4z8423z406bzbeefz8f030a7ceea6" 
   isLoaded="true" isAbstract="false">
        <mdfmetadata xsi:type="conceptorshapemdfmetadata" 
           accessmodifier="public" category="" 
           description="" doccomment="" localize="false" />
              <valueProperties>
                 <property name="Description" 
                   identity="e19a67e2-f2fe-4778-9ef5-799ebdd49945" 
                   id="ie19a67e2zf2fez4778z9ef5z799ebdd49945">
                     <mdfmetadata xsi:type="propertymdfmetadata" 
                       accessmodifier="public" category="" 
                       description="" doccomment="" 
                       localize="false" xpathexpression="" 
                       reversexpathexpression="" realattributename="" 
                       proxyattributename="" customstorage="false" 
                       allownulls="false" readwritestate="readwrite" tag="" />
                     <propertyInfo xsi:type="StringPropertyInfo" 
                       name="" identity="0c3d7e61-0733-4eb5-9a07-b47ae8891135" 
                       default="" max="2147483647" />
                 </property>
              </valueProperties>
</concept>

Next, he wishes to define how these classes interact. He needs to define certain relationships. The meta-data here is defined by many different variables, but the idea is that an object called PageFlow contains as its children, a collection of Page objects. Hence the name PageFlowHasPages. The <role> node defines, by using GUIDs and some aggregation attributes, which child objects the parent has a relationship with. The template at this stage is not worried about how many children there are; this is just a definition describing a relationship. The actual implementation would exist in the generated project from the results of this schema project. In other words, this relationship is just telling the output project (where the actual classes will be created and the functional code applied) that the PageFlow can have multiple Page objects as its children. Notice too that the relationships are keyed by GUIDs to prevent collisions:

<relationship name="PageFlowHasPages" 
    identity="74c80045-e338-4ece-b8d3-96800cf0f565"
    namespace="Fabrikam.UIPChart.DomainModel" 
    id="i74c80045ze338z4ecezb8d3z96800cf0f565" 
    isLoaded="true" isAbstract="false">
    
    <mdfmetadata xsi:type="mdfclassmetadata" 
         accessmodifier="public" category="" 
         description="" doccomment="" localize="false" />
    
    <roles>
      <role name="Pages" 
        identity="e5d3011e-1eb2-4ad1-b365-5dda529ca0d9" 
        id="ie5d3011ez1eb2z4ad1zb365z5dda529ca0d9" 
        min="0" max="0" isUnbounded="true" accepts="all">
      <mdfmetadata xsi:type="rolemdfmetadata" 
         accessmodifier="public" category="" description="" 
         doccomment="" localize="false" 
         isordered="true" isnavigablefrom="false" />
           <source>i17c43f6bz9bdcz4f78za245z13c7b8357e5a</source>
           <type>ibc358cb4z8423z406bzbeefz8f030a7ceea6</type>
           <generatedProperty name="Pages" 
               identity="43b41ecd-e9d2-428a-91af-3c2327a699fc">
           <referenceType>ibc358cb4z8423z406bz
                   beefz8f030a7ceea6</referenceType>
           </generatedProperty>
      </role>
      <role name="PageFlow" 
           identity="6d824167-ed0f-4e41-876f-de95d6b00956" 
           id="i6d824167zed0fz4e41z876fzde95d6b00956" 
           min="1" max="1" isUnbounded="false" 
           accepts="none">
       <mdfmetadata xsi:type="rolemdfmetadata" 
         accessmodifier="public" category="" 
         description="" doccomment="" 
         localize="false" isordered="true" 
         isnavigablefrom="false" />
           <source>ibc358cb4z8423z406bzbeefz8f030a7ceea6</source>
           <type>i17c43f6bz9bdcz4f78za245z13c7b8357e5a</type>
           <generatedProperty name="PageFlow" 
                 identity="e2e62234-8033-4890-bbe2-953bfb964a24">
             <referenceType>i17c43f6bz9bdcz4f78z
                   a245z13c7b8357e5a</referenceType>
           </generatedProperty>
      </role>
    </roles>
</relationship>

The architect will also need to set up how each Page object in Pages will interact. This is also how he sets up the validity of these interactions. Each relationship object or Transition will allow certain interactions between these objects. Notice the min/max attributes below. They indicate levels of aggregation that are acceptable for the relationship. Again, these relationships are keyed together by GUIDs to prevent collisions:

<relationship name="Transition" 
  identity="ecedf22b-3610-4438-80b2-2815df88ec08"
  namespace="Fabrikam.UIPChart.DomainModel" 
  id="iecedf22bz3610z4438z80b2z2815df88ec08" 
  isLoaded="true" isAbstract="false">
    <mdfmetadata xsi:type="mdfclassmetadata" 
       accessmodifier="public" category="" 
       description="" doccomment="" localize="false" />
    <valueProperties>
    <property name="Label" 
      identity="db65ec62-9764-413a-a15f-94303303d25f" 
      id="idb65ec62z9764z413aza15fz94303303d25f">
            <mdfmetadata xsi:type="propertymdfmetadata" 
                accessmodifier="public" 
                category="" description="" doccomment="" 
                localize="false" xpathexpression="" 
                reversexpathexpression="" 
                realattributename="" proxyattributename="" 
                customstorage="false" 
                allownulls="false" 
                readwritestate="readwrite" tag="" />
            <propertyInfo xsi:type="StringPropertyInfo" 
              name="" 
              identity="00000000-0000-0000
                       -0000-000000000000"
              default="" max="2147483647" />
    </property>
    </valueProperties>
    <roles>
<role name="TransitionsTo" 
  identity="6ee7128f-5cf4-4041-92fe-74d4d8efa53e" 
  id="i6ee7128fz5cf4z4041z92fez74d4d8efa53e" 
  min="0" max="0" isUnbounded="true" 
  accepts="none">
        <mdfmetadata xsi:type="rolemdfmetadata" 
          accessmodifier="public" 
          category="" description="" 
          doccomment="" localize="false" 
          isordered="true" isnavigablefrom="false" />
        <source>ibc358cb4z8423z406bzbeefz8f030a7ceea6</source>
        <type>ibc358cb4z8423z406bzbeefz8f030a7ceea6</type>
        <generatedProperty name="TransitionsTo" 
          identity="89dd6aa6-550b-47e3-a184-a1c647d4ce3b">
          <referenceType>ibc358cb4z8423z
                   406bzbeefz8f030a7ceea6</referenceType>
        </generatedProperty>
</role>
<role name="TransitionsFrom" 
   identity="87100260-3e95-4aeb-97ab-edd155f1cb63" 
   id="i87100260z3e95z4aebz97abzedd155f1cb63" 
   min="0" max="0" isUnbounded="true" 
   accepts="none">
        <mdfmetadata xsi:type="rolemdfmetadata" 
           accessmodifier="public" category="" 
           description="" doccomment="" 
           localize="false" isordered="true" 
           isnavigablefrom="false" />
        <source>ibc358cb4z8423z406bzbeefz8f030a7ceea6</source>
        <type>ibc358cb4z8423z406bzbeefz8f030a7ceea6</type>
        <generatedProperty name="TransitionsFrom" 

          identity="510cb874-9b71-4205-90fa-03f961ce453c">
          <referenceType>ibc358cb4z8423z406b
                zbeefz8f030a7ceea6</referenceType>
        </generatedProperty>
</role>
</roles>
</relationship>

How the actual relationship between transitions is set up exists in another section of the meta-data. Here we see the Pages and TransitionsTo objects are established as embedding to identify that the Pages are the noun part of the relationship (see AOM relationship trees for more detail on this concept) and the reference is part of the action or verb part of the relationship:

<treenavigator name="intrinsic" 
  identity="f4536c62-0f9f-4972-bb56-a08b3f068dac" 
  id="if4536c62z0f9fz4972zbb56za08b3f068dac">
    <expressions>
    <treeExpression xsi:type="RoleExpression" 
        name="ModelHasClasses.Classes" 
        identity="ccf8f69f-9855-40fc-a0ac-fb74452e5f71" 
        id="iccf8f69fz9855z40fcza0aczfb74452e5f71" 
        definitionlevel="none" 
        containmentstyle="embedding">
            <source />
            <role>ie5d3011ez1eb2z4ad
                   1zb365z5dda529ca0d9</role>
    </treeExpression>
    <treeExpression xsi:type="RoleExpression" 
        name="ExampleRelation.Target" 
        identity="a48df652-7071-4f17-9c95-4042e1b46aa7" 
        id="ia48df652z7071z4f17z9c95z4042e1b46aa7" 
        definitionlevel="none" containmentstyle="reference">
            <source />
            <role>i6ee7128fz5cf
                   4z4041z92fez74d4d8efa53e</role>
    </treeExpression>
    </expressions>
    <roots />
</treenavigator>

So now, the architect has set up all the possibilities for how Widget A will work. He has limited Widget A to only a small subset of functionality, which is OK, because that's all Widget A needs to do. He has defined all possible interactions and associations that can be implemented in the actual code level project.

Now, let's talk about how the actual workflow in the output project works and how we code against this template.

Software Factory Workflow

In a software factory model, workflows are used as definitions. How the relationships occur is defined in the factory, but it is left to the output project to make sense of this definition and implement code within the definition. Take a Class Diagram DSL project for example. In this type of project, you are setting up class definitions and stating which class types you wish to allow to interact in which ways with other class types. In the Domain Model/Designer project, you have a schema for setting up relationship rules for class types, the class types themselves, and any other rules or attributes pertaining to each. Again, this is just a definition for your actual output project, not a working relationship tree between entities.

Notice the diagram below. We see the actual UML representation (although according to Microsoft, this is not UML) of the same relationships we saw in the XML above. This is, in fact, the UI representation of the domain model designer meta-data we were looking at above.

We see PageFlow with its relationship (one to many) with Page(s) and the relationship TransitionsTo defined as (many to many).

TransitionsFrom is hidden, but is represented by the right side block with an asterisk (*) inside it. We also see a definition for the Transition relationship object and attributes hanging off all the objects (Description and Label):

We can see another view below of the same pattern, listing class types with their underlying relationship trees. If you notice, the PageFlow class has a collection of Page classes as its children, and each Page class has a relationship object (called a Reference property) which aids to define which other objects it needs to interact with. There are also the Roles and Value properties. The cardinality of the tree is maintained a lot like AOM relationship trees by allowing the reference property to handle the relationship definitions. This is a good practice for relationship trees, as it gives more flexibility to the entire tree and preserves the cardinality, both up and down the tree.

OK, so we could infer by the nature of the relationships we have seen so far that there is a workflow here. PageFlow contains groups of Pages. Each Page object can be related to any other Page object as either a parent or a child role. In the Domain Model project, all this can be set up before any actual coding occurs.

Now, let's look at our actual output project and see how these rules influence the code our developers write. First, we are going to look at a UI which will allow us to build class relationships. In their code examples, all this is rather primitive, and it is hard to see how this will translate to actual functional projects. But leave it to Microsoft, they will get this right eventually. I believe they are just showing us the concept here.

When we look at the project toolbox, you can see there are special controls just for your output project type.

If you remember these, map right back to the project definitions we set up in our Domain Model/Designer project. PageFlow is not seen because it is a container object, and for our purposes, interaction is handled automatically. We can add a couple of pages to the actual project workflow through the nifty GUI editor they provide. We can also add relationships (Transitions). If we had another class here that didn't have Transition as a relationship, and tried to add a relationship object between that class and Page, we wouldn't be able to.

We can see the class relationship in the Explorer interface. Notice that Pageflow contains Pages, and each Page is listed below that tree.

If we look at the Report Template for an example of the output project code, we can see how we might write code to reflect our pre-defined class definitions. We see we can loop through a collection of Pages in PageFlow. We also see we can retrieve Pages from each Transition relationship, which infers that cardinality is maintained by this object:

Usefulness of the Methodology

Now comes the big question. Why is this useful? Well, let's say that we had a group of objects from which we have always coded the exact same way, with the exact same rules. Our developers tell us that for each project of this type, they are always doing the same things over and over. They also say that sometimes someone forgets the rules and hoses up the whole operation, resulting in costly production outages. So, you, as an architect, might use the software factory method to build some of these rules into the actual project from which the developers develop. That way, if anyone violated the rules, the project simply wouldn't compile. Another example might be that projects always use the same resources, whether they are web services, database linkages, or code assemblies. Why not create a project type that automatically imports and defines the relationships for these resources? Well, I imagine there are a thousand other uses for this kind of template driven methodology. Just think of it as another refinement for how you write code for a specific business.

The problem with abstract models is that there is not enough agreement on how those abstracts should be rendered in a concrete world. One development shop sees the idea one way, and another sees it quite differently. Developers and teams of developers, in general, would have to start agreeing as to what the approaches for specified problems would be...i.e., abstractions for Data, Business, and UI components. I bet you would be hard pressed to find a consensus between a wide group of developers regarding even the need for those three layers!

So, getting an abstract model to a concrete one is not the real problem. Deciding not to reverse engineer or meddle outside the modeler scope with code is not the real problem. The problem is getting a wide enough acceptance and use of a modeler to make it into something other than just a neat way to show that as developers we can understand abstraction and modeling! It would take a general consensus among developers and IT business leaders to do this. I am not saying it is impossible, and eventually it may happen, and should happen. The problem we are simply faced with is that it is not going to happen easily or overnight. We first need to address the problems of why people will go around and even abandon such creations. And, those problems lie with the fact that even as tech companies like Microsoft are making great strides ahead toward that end, we are still in the infancy of such technology. Not because it is new. The DSL tool examples are not a new idea, just a re-thought one. MS always had domain specific projects, just in very basic domains they felt would be widely used. What we are going to have to work on before such modeling tools are widely used and accepted are the basic problems behind their failure in the past:

1. Usability - ease of use by developers; if it takes twice as long to use the modeler than to write the code, it fails. If the complexity of the tool exceeds more than 20% of its added features or usefulness, it fails. If the developer simply can manually code faster, even if the code is of poor quality (as the majority of code is), then the tool fails. If the tool cannot easily be understood and interpreted, it fails.
2. Scalability - if you cannot easily add and subtract the needed elements into the modeler, then you have limited its usefulness. The scope of the things you can add or subtract should be wide open to new and ever evolving partitions and categories.
3. Abstractive Intuitiveness - either the modeler has to understand better definitions of rules, relationships, and processes of components, or has to be flexible enough to have these added on the fly. Again, we can refer back to the first item above, because if in doing this, usability is compromised, we are back to nothing.
4. Learning Curve to Operate - the tool has to be simple enough to not add to the overhead greatly of the ability of its target audience. If a high school kid learning the tool cannot easily operate it and perform basic tasks, then it is useless (not to down on high school kids, they are as smart as we old farts are!).
5. Abstraction does not make Obfuscation - the level of abstraction is still close enough to the concrete world to make it easily interpretive. In other words, if you abstract up to too high a level, you will lose your users. A reader compared this to going from assembly to Visual Basic and that we would not go back and write assembly. Why? Because assembly is much harder to write code and decipher/debug in. People tend to gravitate to what is easier, not what is harder. So such a modeling tool must be an asset, not overhead.

One thing to think about here is what would a real development cycle be like. We need to think, are we adding complexity? Well, since we are adding another development cycle here, yes we are. So what is the payoff? Well, you could argue that having projects with predefined relationship definitions allow more productivity and flexibility to fall into the hands of business developers. Domain specific experts also could be hired to write software factories specific to a process, which could then be turned over to development teams for actual development. Certain functionalities could be templated for a specific process, and that could shorten development time.

What if you could build a project specific to your business type? You could then sell this project to other businesses who could develop inside the pre-determined business scenario you have defined. But you would have to have figured out all the relationship definitions for that particular business type.

What if you, as an architect, could load use case models into your development project and create the project limitations or scope before development occurred? You could flesh out all the specific problems and limitations to your individual business problem before your developers even began to program, potentially saving money in wasted development costs.

The real thing I think we should look at though is the reality of business development right now and what would have to change. Right now, when developers start development, they don't have to worry about the limitations or scope of their code right away. If suddenly they had to plan and develop the problem before they started coding, then that adds a layer of complexity right from the start. What if their project manager came up a week into the project and asked them where they were and what they had accomplished? They might say, "well, we are still building the definition of the project..." The project manager would then say "'but we already did that in business analysis!" and get worried the developers were not moving efficiently.

I guess what I am getting at is for software factories to work in the real world, people's ideas about software needs to change. Businesses would have to be run a little more efficiently or hire outside companies to come in and template their business processes, so that their development teams would not be bogged down with that job and could concentrate on production code. That might make businesses with proprietary business processes a little nervous. So then, that would require that their business developers would have to now do architecture work. I could not count the number of businesses out there that don't even have an architect, and whose business experts are not developers but managers and non-technical personnel.

Before software factories can come of age, certain human factors need to change. This is what I see. I would like to hear from anyone with anything interesting to say about this subject. To close, I would say we have not even addressed these issues to a satisfactory amount to start saying we have the cure for the common (code) bug . What we do need to do is solve these problems in the market place, where the reality of the business of software is really forged. That ground has yet to be trod. I do hope to see it so in the near future.