Enhanced Configuration File Handling III

• Download source code - 316 KB

Succeeding Article

Enhanced String Handling

What is it all about

A configuration file entry may contain a variable, an if-logic construct, and now it may even contain entries that change "on the fly", dynamically. An "on the fly" or dynamic variable may change every time the config variable is accessed. This article presents a solution for accomplishing these tasks.

Introduction

An appSettings section of a config file contains literal keys coupled with literal values. Having literals (as opposed to variables) is done so by design, as the (key, value) pair is intended to keep a place holder for initialization, static initialization, usually between runs of a given program. Nevertheless, at times, it is more comfortable (and less hassle) to use variable specifications and logic as opposed to using static literals. So, for example, in the appSettings of a config file, we may use the following:

<add key="BaseDir" value="c:\somedirectory"/> 
<add key="TestFile" value="{key::BaseDir}\FileName"/>

instead of:

<add key="TestFile" value="c:\somedirectory\FileName"/> 

If the directory "c:\somedirectory" is used in multiple entries of the appSettings, then using {key::BaseDir} will be a clear win, especially if {key::BaseDir} corresponds to a value that changes from time to time.

Motivations and goals

Main Motivation: Ability to use config variables within a configuration file. The use of config variables makes configuration files more comfortable to use; if-logic variables add pleasure to the comfort; and now config variables that can change dynamically add an ecstatic thrill experience to the use of configuration variables. If you think that you would rather look for your ecstatic thrills elsewhere, try to understand the logic within this article and the accompanying code, then decide.

Known to the User Only: Another motivation is to provide the end user with an additional comfort by allowing the user to specify new config-entries. These new entries are neither mandatory nor known to the system up front.

The system expects some mandatory and optional config entries, without which the system may either consider the config-file in an error state, or the system may supply default values for these missing entries.

New entries, unknown to the system, are not necessarily an error, though the system cannot make use of them internally. At times, it is convenient for the user to specify a new config entry (not known to the system) and thereafter make use of the new entry as a shortcut within a mandatory config entry. These "known to the end-user only" entries may contain variables and conditional entries. As an example, consider the following config file:

<configuration>
   <appSettings>
       ...
       <add key="Flat" value="testing"/>
       ...
       <!-- temp is inaccessible programmatically, known to the user only -->
       <add key="temp" value="SpecialDirectory\{key::flat}"/>

       <add key="BaseDir" value="c:\somedirectory"/>
       <add key="Current path" value="{key::BaseDir}\{key::temp}"/>
       ...
   </appSettings>
</configuration>

The system expects the following config entries:

• Key="Flat"
• Key="BaseDir"
• Key="Current path"

The system knows nothing about the config-entry Key="temp", and as such, the system cannot make use of it internally. Nevertheless, the user knows about "temp" and can take advantage of its existence in other entries, like in: Key="Current path", and all is well.

Convention: A convention that serves me well is: entries in a config file that the system expects are entered in Pascal casing, and entries that are unknown to the system are entered in Camel casing. (Pascal casing: first letter of the identifier is capitalized, and Camel casing: first letter of the identifier is lower cased.)

Case Insensitive: Another goal is to treat and match the keys in the entries in a case insensitive manner. So, <add key="flat" value=".."/> and <add key="FLAT" value=".."/> are one and the same entry. Also, either {key::flat} or {key::fLaT} will find the above config-entry. During coding, we will need to be vigilant that our string comparison will be case insensitive and our Regular Expressions will include the RegexOptions.IgnoreCase option. This case sensitivity sentiment is shared by ConfigurationManager.AppSettings[key], where key can be given in a case insensitive manner.

Nesting / Sequential: Config variables may be entered nested within other config variables or entered side-by-side. So for example, in the following entry:

<add key="K0" value=".."/>
<add key="Condtn" value=".."/>
<add key="T" value=".."/>
<add key="F" value=".."/>
<add key="K1" value="{key::K0}..{if ({Key::condtn}=ab) {Key::T}, {key::F}}"/>

{key::K0} is entered side-by-side to the {if ..} construct, while {Key::condtn} is nested within the {if ..} construct.

Forward / backward referencing: A {key::name} variable should be able to reference an entry that was read already or an entry that will be read. For example:

<configuration>
   <appSettings>
       ...
       <add key="Test Forward Referencing" value="{key::BaseDir}\WillFollow"/>
       <add key="BaseDir" value="c:\somedirectory"/>
       <add key="Test Backward Referencing" value="{key::BaseDir}\AlreadyRead"/>
       ...
   </appSettings>
</configuration>

Multiple store support: The algorithm described within needs to operate on any (key, value) pair store, not just the appSettings section of a configuration file. The store needs to be any permanent store of a (key, value) pair collection. For example, the code structure will support a flat file containing key=value entries. Therefore, I have replaced the word "config" with "ConcordPair" in the DLL, which is agnostic of the type of the store providing the (key, value) pair collection.

Extendible: The system should be able to extend itself to support more ConcordPair evaluation constructs. I leave it as an exercise for you to provide the design and coding support for the constructs {Intelligence::read mind::person-name} and {Intelligence::change mind::person-name}.

Windows or Web, C# or VB

The code example provided with the article is a Windows console application written in C#, but nothing prevents you from using the concept, as is, in a Windows Forms application, WPF application, in an ASP web environment with or without WPF, or translating it to VB or any other language.

A very high level view

There are five (5) major parts to the solution:

• The ConcordPairEval class, part of ConcordPairEvaluate.DLL, that is responsible for the static and dynamic variable evaluation. The methods primarily responsible for the evaluation of static variables are EvaluateConcordPair() and EvalSimpleExpression(). Dynamic variables are evaluated primarily within the EvaluateOnTheFly() method.
• The repertoire of ProcessXxx classes, part of the DLL, each one of which handles a variable kind. The ProcessIntelligence class supporting the {Intelligence::..} construct is not provided.
• ConcordPairOnTheFlyGlobal, a class part of the DLL, houses state information variables. These state variables are used by the dynamic variables to report the on-the-fly values.
• The Config class, part of TestOnTheFlyConfig.EXE, that is responsible for:
  • Initialization in the Instantiate() method, where the ProcessXxx are added to two delegates used to evaluate ConcordPair (or config) variables. One delegate is used for static variables evaluation and one for dynamic variables evaluation.
  • Reading the store in its entirety, also part of the Instantiate() method.
  • Get (or, less frequently, get and set) property for each ConcordPair entry.
• Usage of the ConcordPair entries, as exemplified in the Main() method.

If all you want to do is to move on down the road

If understanding how things are done is secondary to solving the problem at hand, then:

• Copy the DLL ConcordPairEvaluate.
• Modify or add to the repertoire of ProcessXxx classes. Each class handles a different evaluation that you need.
• Modify the ConcordPairOnTheFlyGlobal class to your needs, for dynamic evaluation constructs.
• Copy the Config class from the EXE, TestOnTheFlyConfig, and modify it to your needs.
• Mimic the access to the config variables as shown in the Main() method of the Program class.

Hereafter, you are on your way. I would recommend that you go back and read the rest of the article at your leisure.

Review of previous articles

The first article, Enhanced Configuration File Handling, in this trio of articles, deals with the ability to specify static variables, for example, {key::name}, {date::yyyymmdd}, {ForeignKey::path-of-external-file::value} etc. The first article’s accompanying code ensures that variables can exist side by side.

The second article, Enhanced Configuration File Handling II, in this trio of articles, extends the capabilities described in the first article by adding if-logic to the config entries. If-logic variables are considered to be static variables. The second article’s accompanying code ensures that in addition to side by side existence, variables can coexist as nested.

This article, the third article in the trio, takes the static variables to the next level, dynamism. The article, and also the accompanying code, make a distinction between the generic DLL processing and the specific Config class processing in the EXE. Generic—in the sense that the DLL is agnostic of the type of store it supports. Specific—in the sense that the Config class, in the EXE, represents a specific store, in this example—a configuration file.

Should you read the previous articles to understand this one?

No! Despite the fact that this article is a continuation of the previous two, I will describe the difficult parts of the source code. The code accompanying this article is a bit different than the code presented in the previous two articles. However, reading the previous articles is of value. For example article 2, Enhanced Configuration File Handling II, gives a concrete example as to the reason for having “{literal-string}” evaluated to “literal-string”. Both articles explain some of the decisions I took along the way.

Variable-Evaluation provided with the accompanying code

Each variable type is handled within a separate ProcessXxx class residing within the ProcessEvaluate subdirectory of the DLL. All the keys in the variables are interpreted in a case insensitive manner. Some of the values following the keys are case sensitive and some are not, I will itemize which is which.

{Key::Name}: Where it all started from. A static variable provides the ability to evaluate to another key-entry’s value. The variable {key::BaseDir} refers to another entry in the store having its key=“BaseDir”. Thereafter, the system will fetch the value of “BaseDir”. See the following config snippet:

<configuration>
   <appSettings>
       ...
       <add key="BaseDir" value="c:\somedirectory"/>                  (1)
       <add key="SubDirectory" value="{key::BaseDir}\someSubDirectory"/>       (2)
       ...
   </appSettings>
</configuration>

{key::BaseDir}, in line 2, evaluates to “c:\somedirectory” coming from line 1, making the final value of key="SubDirectory" be “c:\somedirectory\someSubDirectory”.

The literal “key”, in {key::BaseDir}, is case insensitive, and so is the name search for “BaseDir”, so {KEY::basedir} would have evaluated, in the above example, to “c:\somedirectory” as well.

{Date::format}: Statically evaluates to today’s date and renders the date according to the format provided. The valid format components are:

• “yyyy” and “yy” as 4 and 2 digit year (lower case).
• “MM” and “M” as 2 and 1 or 2 digit month (note that “MM” and “M” are capitalized, as this will be consistent with the format in {CurrentTime::format} (upper case).
• “dd” and “d” as 2 and 1 or 2 digit day of month (lower case).

So, for example: {Date::yyyyMMdd} will evaluate to 25251121 if today’s date is the 21^st day in November of the year 2525, and {Date::MM/dd/yyyy} will evaluate to 11/21/2525. Note that {Date::format} is a static evaluation and as such will not change even if the calendar day changes while the program is running.

Note that the key, or the literal “date” specification, is case insensitive, the format of the variable is case sensitive!!! So, in the above example (giving a lower case “date”), {date::yyyyMMdd} will still evaluate to 25251121, but {date:yymmdd} will evaluate to 2525mm21 as the system does not know what to do with “mm”, it knows what to do with “MM”.

{ForeignKey::path::name}: Statically evaluates a {key::name} pair residing in a flat file composed of key=value pairs. The flat file is provided by the path. The path may be a full path given in a UNC format or drive-letter colon format, or the path may be a relative path, in which case the relative path will be rooted in the path of the executable. (The executable path will be prefixed to the path in the variable to form a full path.)

The literal key, “ForeignKey” and the “name” in the above variable are processed in a case insensitive manner, the path in Windows is always case insensitive as well.

{if (condition) tExp, fExp}: Statically evaluates an if-logic variable. The condition is a choice between three (3) options:

• DirectoryExists(directory-name): Where “directory-name” may be a variable expression and it must be a full path. The literal “DirectoryExists” is searched for in a case insensitive manner.
• FileExists(file-name): Where “file-name” may be a variable expression and it must be a full path. The literal “FileExists” is searched for in a case insensitive manner.
• Exp1 = Exp2: Where both “Exp1” and “Exp2” may be variable expressions. Exp1 and Exp2 are evaluated for equality in a case insensitive string comparison.

tExp and fExp: tExp is the expression to be evaluated as the value of the {if ..} expression provided that the condition evaluates to true. fExp, by contrast, is the expression to be evaluated as the value of the {if ..} expression provided that the condition evaluates to false. (As you can imagine the “if” key specification is case insensitive, so {if ..} and {IF ..} are equivalent.)

{CurrentDir::}: Dynamically evaluates to the current directory in the processing of the system. For the purposes of evaluation, the system fetches the value of the “current directory” from ConcordPairOnTheFlyGlobal. The literal key “CurrentDir” is specified in a case insensitive manner.

{CurrentTime::format}: Dynamically evaluates to the current time and renders according to the format provided:

• “yyyy” and “yy” as 4 and 2 digit year
• “MM” and “M” as 2 and 1 or 2 digit month
• “dd” and “d” as 2 and 1 or 2 digit day of month
• “hh” and “h” as 2 and 1 or 2 digit hour in a 12 hour clock
• “HH” and “H” as 2 and 1 or 2 digit hour in a 24 hour clock
• “mm” and “m” as 2 and 1 or 2 digit minutes
• “ss” and “s” as 2 and 1 or 2 digit seconds
• “fff” as 3 digit milliseconds
• “ff” as 2 digit hundredth of a second
• “f” as 1 digit tenth of a second
• “tt” for “AM”/“PM”
• “t” for “A”/“P”

For example: {CurrentTime::yyyy.MM.dd hh:mm:ss.fff tt} will yield 2525.11.21 12:23:02.208 AM, if the date currently is: 11/21/2525 12:23:02.208 in the morning. (The literal “CurrentTime” specification is case insensitive.)

{literal}: Statically removes the braces off a literal string. The string may not contain a double colon character set, nor may it start with an “if”.

Typographical limitations

By employing the solution provided, you cannot use braces or double colon character sets freely, they have meaning within the framework, and the ConcordPairEval class insists that they be balanced.

The {if ..} construct uses a comma, an equal sign (“=”), and parentheses, which automatically limits your use of these characters within the {if ..} construct.

These limitations are theoretical limitations more than practical ones. I itemize them here for completeness.

Nomenclature

ConcordPair Variable: A variable having an unspecified store type. A config variable, by contrast, is a variable specification in a configuration file—its store type is configuration. A ConcordPair variable and a config variable represent the same entity, for example {key::name}, but a ConcordPair variable concept affords us the freedom to talk about these variables without having ourselves pinned down to a store type.

Static vs. Dynamic: I will refer to the "on the fly" variables as dynamic variables. In contrast, I will refer to the "regular" ordinary every day variables (like the ones described in the first and second articles) as static variables. A static ConcordPair variable does not change throughout the life of the program. A dynamic ConcordPair variable may potentially change at every access of the ConcordPair variable.

A dynamic variable may be nested within a static {if ..} variable. This will be resolved by the dynamic evaluation processes that resolves static left over variables as well as dynamic variables.

Simple expression: A simple expression is one that has no inner expressions. For example, {key::MyKey} in the following MyNextKey config entry is a simple expression. The {if ..} construct in the following M2 config entry is not simple, since it contains other expressions in its midst.

...
<add key=MyKey value="What a wonderful world"/>
<add key="MyNextKey" value="Simple expression: {key::MyKey}"/>
...
<add key="M2" value="{if ({CurrentTime::mm} = 00) Top of hour, not yet}"/>
...

Key / body: A ConcordPair variable is composed of a key. In {key::name} variable, the key is named "key", and in the variable {date::format}, the key is named "date". The key is a mandatory part of a ConcordPair variable. The double colon character set that follows the key is an almost mandatory addendum to the key. Almost, as the {if ..} construct does not have a double colon in it. Everything following the double colon after the key is the body. So, in the case of {key::name}, the body is "name", and in the case of {ForeignKey::c:\path\flatfile.txt::IP of server A}, the body is "c:\path\flatfile.txt::IP of server A". In the case of {CurrentDir::}, the key is "CurrentDir" and it has no body.

ConcordPair-Literal: A "literal" or a ConcordPair-literal is a construct of the form {literal-string} where "literal-string" is a literal. It is evaluated to the literal string itself, provided that the literal string does not have double colon in its midst nor does it start with an “if” followed by parenthetical enclosed characters.

A walkthrough from a client point of view

The code in the “walk through” example makes use of a WriteLine() routine within a LogOut singleton class, for example:

LogOut.Inst.WriteLine(string.Format("{0}  Flat: {1}", tmStamp, flat));

The class can be found in the Utilities folder of the DLL. You need not review it now and detract from your train of understanding the ConcordPair evaluation process. Though I would like to make you aware of three (3) points:

• Point 1: The LogOut class writes to two places, the console and a hard coded file name: C:\Temp\ConcordPairEvaluate.txt.
• Point 2: This additional write requires a Stream close which happens within a LogOut’s Dispose() method. This Dispose() method is called from the Cleanup() method of the Program class.
• Point 3: The LogOut’s WriteLine() method prepends each line with a time stamp: DateTime.Now.ToString() + "\t", for your time comparison convenience.

Back to our walk through business

The sample program is divided into a DLL and an EXE. The EXE has a class Program, and within it, we have a method Main() that, for our purposes, is the client.

static void Main(string[] args)
{
   bool rc = Init();
   if (!rc) { Cleanup(); return; }
   ...

The client, or Main(), starts with a call to an Init() method that performs a configuration check.

The first config entries yielding two lines of output:

<add key="Flat" value="testing"/>
<add key="Stamp" value="{CurrentTime::yyyyMMddHHmmssfff}"/>
<add key="Static flat" value="Static evaluation of flat: {key::flat}"/>

Code… continuing with the body of Main():

...
string flat = Config.Inst.Flat;                   // Static variable
string tmStamp = Config.Inst.Stamp;               // Dynamic variable
LogOut.Inst.WriteLine(string.Format("{0}  Flat: {1}", tmStamp, flat));

string staticFlat = Config.Inst.StaticFlat;       // Static variable
tmStamp = Config.Inst.Stamp;                      // Dynamic variable
LogOut.Inst.WriteLine(string.Format("{0}  Static Flat: {1}", tmStamp, staticFlat));

Yielding:

9/7/2009 5:08:30 PM     20090907170830686  Flat: testing
9/7/2009 5:08:30 PM     20090907170830702  Static Flat: Static evaluation of flat: testing

The next two lines (indicating that StaticDate does not change), an additional config entry:

<add key="Static date" value="Static evaluation of date: {date::yyyy.MM.dd}"/>

Code continuing with the body of Main():

...
string staticDate = Config.Inst.StaticDate;
tmStamp = Config.Inst.Stamp;
LogOut.Inst.WriteLine(string.Format("First:  {0}  Static Date: {1}", 
                      tmStamp, staticDate));

staticDate = Config.Inst.StaticDate;
tmStamp = Config.Inst.Stamp;
LogOut.Inst.WriteLine(string.Format("Again:  {0}  Static Date: {1}", 
                      tmStamp, staticDate));

Yielding:

9/7/2009 5:08:30 PM  First: 20090907170830705  
         Static Date: Static evaluation of date: 2009.09.07
9/7/2009 5:08:30 PM  Again: 20090907170830707  
         Static Date: Static evaluation of date: 2009.09.07

The last two lines (indicating that Current Dir may change):

<add key="Dynamic current directory" value="current dir: {CurrentDir::}"/>
<add key="temp" value="SpecialDirectory\{key::flat}\{key::Stamp}"/>
<add key="Dynamic current path" value="{key::dynamic current directory}\{key::temp}"/>

Code continuing with the body of Main():

...
ConcordPairOnTheFlyGlobal.Inst.CurrentDir = @"C:\";
LogOut.Inst.WriteLine(string.Format("First:  Current dir: \"{0}\",  
 current path: \"{1}\"", Config.Inst.DynamicDir, Config.Inst.DyanmicPath));

ConcordPairOnTheFlyGlobal.Inst.CurrentDir = @"C:\Another Directory";
LogOut.Inst.WriteLine(string.Format("Again:  Current dir: \"{0}\", 
 current path: \"{1}\"", Config.Inst.DynamicDir, Config.Inst.DyanmicPath));

Yielding:

9/7/2009 5:08:30 PM     First:  Current dir: "current dir: C:\", 
 current path: "current dir: C:\\SpecialDirectory\testing\20090907170830713"
9/7/2009 5:08:30 PM     Again:  Current dir: "current dir: C:\Another Directory", 
 current path: "current dir: C:\Another Directory\SpecialDirectory\testing\20090907170830715"

The code ends with a cleanup:

   ...
   Cleanup();
}

How does the code perform an evaluation?

From a high level overview—there are two sides to handling the dynamic evaluation:

• Evaluation of the static variables once. The evaluation happens before any static variable is used.
• Evaluation of the dynamic variables that occurs at every reference to the ConcordPair variable. This evaluation happens at the access point of the variable. Code-wise: the system defines a singleton class that houses variables, each one of which tracks one of the states of the system. The dynamic evaluation reports that state.

Both static and dynamic evaluations have parts that are generic, therefore they are tucked away in a DLL, ConcordPairEvaluate; and parts that are store-specific and as such are not in the DLL. The DLL strength is the fact that it is agnostic to the type of store that it operates on.

From a high level point of view, a ConcordPair variable is dealt with as follows:

• System reads the ConcordPair store in its entirety.
• System stores all the entries in an internal structure, whether they are mandatory or not.
• System checks for any errors in the store.
• System resolves the static variables.
• On every access of a ConcordPair entry, the system will resolve any dynamic variables by extracting state variable information from a separate class (in our example: ConcordPairOnTheFlyGlobal).

Before looking at the code that accomplishes all this, let’s look at the architecture layout of the classes involved.

Architecture of classes

Review of classes

ConcordPairElement: The ConcordPairEval class utilizes get and set methods, and avoids dealing with the underlying store altogether. These get and set methods build and retrieve a ConcordPairElement class. The signature of the get and set methods is as follows:

protected virtual ConcordPairElement GetConcordPairElement(string key)
protected virtual bool SetConcordPairElement(string k, ConcordPairElement e)

ConcordPairEval: The diagram starts with this class. It houses the entries of the store in a private dictionary, _concordPairEntry, of type Dictionary<string, ConcordPairElement>. It is a base class for the Config class.

You may notice that the entries of the store could have been housed in a Dictionary<string, string> where the first string generic type of the Dictionary houses the key and the second generic type houses the value of a (key, value) pair. ConcordPairElement contains both the key and the value of an entry, therefore a Dictionary<key, ConcordPairElement> contains an extra “key”. This extra key in the Dictionary comes to resolves the case sensitivity issue. The “extra” key is stored in lower-case always.

The ConcordPairEval class is responsible for evaluating the static variables up to the point where dynamic evaluation can take over.

In a sequence diagram, the system reaches ConcordPairEval after the system reads the store (config file, flat file, etc.) in its entirety and had initialized the delegates that are used by the ConcordPairEval methods.

IProcessEvaluate: Provides an interface for the classes that evaluate the ConcordPair variables, be those variables static or dynamic variables. Those evaluation classes are named ProcessXxx, where Xxx is replaced with a specific name.

ProcessXxx: This is where the evaluation magic takes place. The evaluation reduces to a string-replace in these classes. These classes follow a template, making it easier for you to create new ones. This ability ensures that the solution is extendible.

Each of the ProcessXxx classes evaluates a specific ConcordPair variable. Each of the ProcessXxx classes implements the IProcessEvaluate interface. The IProcessEvaluate interface and the delegate that invokes the ProcessXxx classes use the same signature—obviously. The OnStaticEvaluateHandler delegate as well as OnDynamicEvaluateHandler in the ConcordPairEval class are linked to the Evaluate() method of each of the ProcessXxx classes. The accompanying example provides seven (7) ProcessXxx classes:

• ProcessKey class processes the {key::name} variable.
• ProcessDate class processes the {Date::Format} variable.
• ProcessForeignKey class processes the {ForeignKey::path::name} variable.
• ProcessIf class processes the {if (condition) tExp, fExp} variable.
• ProcessLiteral class processes a {literal-string} construct to “literal-string”.
• ProcessCurrentDir class processes the {CurrentDir::} variable.
• ProcessCurrentTime class processes the {CurrentTime::Format} variable.

These are by no means exhaustive, but they are a good start.

ConcordPairEvalEventArgs: The ConcordPairEval class uses two event delegates that use the ConcordPairEvalEventArgs.

ConcordPairOnTheFlyGlobal: Is the class that keeps track of state variables which are required by the dynamic variables.

Config: The Config class is responsible for:

• Initializing the delegates that will be used by ConcordPairEval to call the ProcessXxx.Evalute() methods.
• Reading the store (in our case, the store is a config file).
• Checking for errors.
• Providing a get property for each of the mandatory config entries.

Functional point of view

Reading from the config file

The Config class constructor calls Instantiate() that reads the appSettings section of the config file. Reading the appSettings of App.config could have been accomplished like so:

Dictionary<string, string> configEntries = new Dictionary<string, string>();
foreach (string key in ConfigurationManager.AppSettings.AllKeys)
   configEntries.Add(key, ConfigurationManager.AppSettings[key]);

Instantiate() does a similar job:

ConfigSucceeded = true;
foreach (string key in ExtractKeys())
{
   Try
   {
      string sBody = ConfigurationManager.AppSettings[key];
      ConcordPairElement eBody = new ConcordPairElement(key, sBody);
      SetConcordPairElement(key, eBody);
   }
   catch (ConfigurationException ex)
   {
      HandleConfigEntryError(key, ex);
      ConfigSucceeded = false;
   }
}

Where ExtractKeys() returns ConfigurationManager.AppSettings.AllKeys and SetConcordPairElement() essentially performs a _configEntries.Add().

SetConcordPairElement() looks like:

protected virtual bool SetConcordPairElement(string key,
                                              ConcordPairElement element)
{
  if (key == null) return false;

  key = key.ToLower();       // Notice the lower case transition
  if (_concordPairEntry.ContainsKey(key))
     throw new ArgumentException(..);

  _concordPairEntry.Add(key, element);
  return true;
}

Store all the entries

All entries are read into the dictionary automatically by the code presented above.

Errors in the config file found

SetConcordPairElement() makes sure that the key is not null.

Instantiate(), before calling EvaluateConcordPair(), calls PreEvaluateConcordPair() and PostEvaluateConcordPair(). PreEvaluateConcordPair() checks that the braces are all balanced.

The Main() method calls an Init() method as the first thing it does, which in turn calls ConfigCheck() that checks for the integrity of the config file.

Beyond which the get properties should check the integrity of the values in the specific ConcordPair entry.

Resolution of static variables and if-logic and dynamic variables

On every access of a ConcordPair entry, the system will use the following template. (I will use the example key=“Flat” as an example.)

private const string _flatKey = "Flat";                (11)
private bool _internalFlat = false;                              (12)
public string Flat                                               (13)
{
  Get                                                            (14)
  {
     // Get the element including any static Config evaluation
     ConcordPairElement elem = GetConcordPairElement(_flatKey);  (15)
     if (elem == null) return string.Empty;                      (16)

     if (_internalFlat)                                          (17)
     {
        // Do the dynamic massaging
        return EvaluateOnTheFly(_evaluateOnTheFlyHandler, elem); (18)
     }

     _internalFlat = true;                                       (19)

     //
     // Massage and Validate elem.Value                          (20)
     //

     return Flat;                                                (21)
  }
}

The first line (line 11) in the entry is the key, _flatKey, having its value as it appears in the appSettings of the config file. Note that it is a const, as opposed to a readonly string, so that I can declare it here and use it anywhere else, even in code appearing prior to its definition, like in the constructor. See the Instantiate() method lines 31-33 below for the code reading the _flatKey config entry.

The second line, marked by (12), is a boolean flag, _internalFlat indicating whether the config entry has gone through static-massaging or not; also see lines 17 and 19 for other instances of usage of the _internalFlat boolean flag.

The property Flat is a get property that starts with a check for the existence of the entry “Flat” in the store (lines 14-16).

If no entry is found in the store, then it returns a default value (line 16). If it is an error for such an entry not to be in the store, then this error would have been caught earlier during the initial processing of the configuration file; _configSucceeded would have been set to false in the Instantiate() method (line 33), and Main() would have stopped (lines 41-44).

The _internalFlat indicates if the static massaging process has happened (line 17). Some of the ConcordPair entries need to be massaged before usage. You will replace line 20 with your code, static-massaging the data before usage.

Line 21 will make a recursive call that will land on line 18 that will dynamic-evaluate the config variable using the method EvaluateOnTheFly(..).

Notice that EvaluateOnTheFly() is called at every get property providing access to a ConcordPair entry. The call is mandatory because up front you do not know if the user will supply the ConcordPair entry with a dynamic variable.

private void Instantiate()
{
   ...
   if (GetConcordPairElement(_flatKey) == null)                      (31)
   {
      HandleConcordPairEntryError(_flatKey, "Key missing");          (32)
      _configSucceeded = false;                                      (33)
   }
   ...
}

static void Main(string[] args)
{
   ...
   bool rc = Init();                                                 (41)
   if (!rc)                                                          (42)
   {
      Cleanup();                                                     (43)
      return;                                                        (44)
   }
   ...
}

A better way: Since EvaluateOnTheFly(..) is called at every ConcordPair get property, an aspect oriented solution could be a better fit than the ubiquitous EvaluateOnTheFly(..) method. You may define an aspect using a framework like Spring.Net or PostSharp Laos or any of the other aspect oriented frameworks. I leave this exercise for you as I am afraid that introducing aspects and an addendum framework here will detract from the understanding of the main purpose of the article.

Review of the code

I will not review the code in its entirety as you can do a better job understanding it if you read the code for yourself. Though, I would like to review the following:

• A ProcessXxx class, and as a representative, I will review ProcessCurrentDir as it evaluates a dynamic variable. It is worth noting that ProcessLiteral is the only member of the ProcessXxx that deviates a bit from the template described below.
• I will review with you the EvalSimpleExpression() method as part of the ConcordPairEval class.
• I will review the EvaluateConcordPair(), also part of the ConcordPairEval class.

ProcessCurrentDir

public sealed class ProcessCurrentDir : IProcessEvaluate
{
  static ProcessCurrentDir()
  {
     string pattern = @"({)\s*CurrentDir\s*::\s*(})";
     RegexOptions reo = RegexOptions.Singleline |
                                          RegexOptions.IgnoreCase;
     _reCurrDir = new Regex(pattern, reo);
  }

  public ProcessCurrentDir()
  {
  }

  ...

  private static Regex _reCurrDir;
}

The class starts by a promise to implement the IProcessEvaluate interface, which means that it will need to implement the Evaluate(..) method. It has two constructors: an instance constructor which is empty, and a class constructor (static constructor) that assigns a value to _reCurrDir, a Regex private class variable. Note, the instance constructor in the ProcessKey class is not empty. It needs a place holder for the routine that will process a {key::name} variable as per the store’s (key, value) entries. The parameter to the constructor of ProcessKey instantiatez the appropriate local delegate to that routine.

The _reCurrDir is the Regular Expression used to identify the {CurrentDir::} variable. Once identified, the Evaluate() method will evaluate it to the current directory.

The Evaluate method

#region IProcessEvaluate Members

public void Evaluate(object src, ConcordPairEvalEventArgs ea)
{
     ea.IsHandled = false;         // Initialize return code
     string text = ea.ConcordPairElem.Value; 
     if (text == null) return;
     if (text == string.Empty) return;
     bool rc = _reCurrDir.IsMatch(text);
     if (!rc) return;

     // The magic happens here
     string replacement = _reCurrDir.Replace(text, CurrDirReplace);     
     if (replacement == text) return;

     ea.IsHandled = true;       // Found {CurrentDir::} and replaced it
     ea.ConcordPairElem.Value = replacement;
     return;
}

#endregion

private string CurrDirReplace(Match m)
{
     return ConcordPairOnTheFlyGlobal.Inst.CurrentDir;
}

The Evaluate(..) method is the one attached to the event delegate, OnDynamicEvaluateHandler, or OnStaticEvaluateHandler. These delegates are declared in the ConcordPairEval class and are initialized by the Instantiate() method of the Config class.

OnDynamicEvaluateHandler += (new ProcessCurrentDir()).Evaluate;

The Evaluate() method serves as the “intelligence” behind the evaluation. The evaluation boils down to a string-replace taking place in the statement:

string replacement = _reCurrDir.Replace(text, CurrDirReplace);    (line 51)

where text, in the above line 51, is the string we want to replace “{CurrentDir::}” with. CurrDirReplace, in the above line 51, refers to the function CurrDirReplace(Match m) (this is nothing more than a straight Regex Replace method).

The following line:

if (replacement == text) return;              (line 52)

looks like it is superfluous. If you look at the CurrDirReplace(Match m) method and the Replace(..) line (line 51), then line 52 can never happen, and as such, it is superfluous. It absolutely is superfluous!!! The only reason I leave it here is because I am following a template for the Evaluate() method, and as such, optimization is of a secondary concern.

EvalSimpleExpression

EvalSimpleExpression(..) is part of the ConcordPairEval class. This method is responsible for the static evaluation of a simple static expression.

Do note that EvalSimpleExpression() is intended to evaluate only one ProcessXxx at a time, and an outer process keeps track of the need for further calls to EvalSimpleExpression(). By contrast, the method EvaluateOnTheFly() is designed to evaluate all the ProcessXxx that it can for a single ConcordPairElement. The static case needs to handle forward as well as backward variable referencing, and I felt that this algorithm may fair better.

The way EvalSimpleExpression() calls the Evaluate() method of each of the ProcessXxx classes is through a delegate, OnStaticEvaluateHandler, which is instantiated by the Instantiate() method of the Config class.

Understanding the EvalSimpleExpression() method will help in understanding the EvaluateOnTheFly() method.

protected bool EvalSimpleExpression(ConcordPairElement elem)
{
  bool rc = IsSimpleExpression(elem.Value);
  if (!rc) return false;

  ConcordPairEvalEventArgs ea = new ConcordPairEvalEventArgs(elem);
  foreach (Delegate del in OnStaticEvaluateHandler.GetInvocationList())
  {
     del.DynamicInvoke(new object[] { this, ea });
     if (ea.IsHandled)
        return true;         // We evaluated something.
  }

  return false;     // We evaluated nothing
}

This routine evaluates static variables only. Only the static delegate, OnStaticEvaluateHandler, is employed.

Point of importance: This algorithm performs simple-expression-evaluation only. We will see shortly that it does so iteratively until the expressions are all evaluated (or the system knows not how to evaluate the variable).

Point 2: OnStaticEvaluateHandler was instantiated with Evaluate() methods before the EvalSimpleExpression() was called. See the Instantiate() method of the Config class.

EvaluateConcordPair

EvaluateConcordPair() is responsible for going through all the entries in the ConcordPair memory (representation of the underlying store) and evaluating the static variables using EvalSimpleExpression().

The method starts by constructing a linked list of all the entries needing attention. A ConcordPair entry needing attention is an entry having an expression that is identified as a simple expression.

LinkedList<ConcordPairElement> concordPairElements;
concordPairElements = new LinkedList<ConcordPairElement>();

IEnumerable<ConcordPairElement> concordPairNodes;
concordPairNodes = 
  from k in _concordPairEntry 
  where IsSimpleExpression(k.Value.Value) 
  select k.Value;

foreach (ConcordPairElement concordPairNode in concordPairNodes)
  concordPairElements.AddLast(concordPairNode);

I employed the linked-list in order to guard against the possibility of having a seemingly simple looking variable that is not recognized by any of the ProcessXxx classes. For example, if we have {Intelligence::read mind::person-name} that has no matching ProcessIntelligence class. Then, despite the fact that IsSimpleExpression(..) returns true, the process leaves “{Intelligence::read mind::person-name}” as is.

We have a limit of PassThroughUpperLimit to our outer evaluation loop. I could not figure out a way to make a key (or a pair of keys) go into an infinite loop of evaluations. Nevertheless, I leave this upper limit in, just as a precaution. Even if an infinite loop is impossible in the currently programmed software machinery, I believe that leaving this upper limit in is a good thing as we need not worry about it when times come to enhance the algorithm.

The rest of the method goes through the linked list, and calls EvalSimpleExpression(), in a loop; if nothing is evaluated, then the method removes the node from the linked list.

public const int PassThroughUpperLimit = 1000;       // Prevent infinite loop
protected virtual void EvaluateConcordPair()
{
   IsEvalConcordPairSuccessful = true;
   if (OnEvaluateHandler == null) return;

   LinkedList<ConcordPairElement> concordPairElements;
   concordPairElements = new LinkedList<ConcordPairElement>();

   IEnumerable<ConcordPairElement> concordPairNodes;
   concordPairNodes = 
      from k in _concordPairEntry 
      where IsSimpleExpression(k.Value.Value) 
      select k.Value;

   foreach (ConcordPairElement concordPairNode in concordPairNodes)
      concordPairElements.AddLast(concordPairNode);

   if (concordPairElements.Count == 0) return;

   for (int i = 0; i < PassThroughUpperLimit; ++i)
   {
      LinkedListNode<ConcordPairElement> node;
      for (node = concordPairElements.First; node!=null; )
      {
         bool bEval = EvalSimpleExpression(node.Value);
         if (!bEval)
         {
            LinkedListNode<ConcordPairElement> p = node.Next;
            concordPairElements.Remove(node);
            node = p;
         }
         else
            node = node.Next;
      }

      if (concordPairElements.Count == 0) break;
   }

   // Check for a loop that exceeded PassThroughUpperLimit iterations
   if (concordPairElements.Count > 0)
   {
      LogOut.Inst.WriteLine(..);
      IsEvalConcordPairSuccessful = false;
   }
}

IsSimpleExpression(..) can be thought of as DoesContainSimpleExpression(..).

protected virtual bool IsSimpleExpression(string concordPairValue)
{
  Match m = _reSmplExpr.Match(concordPairValue);
  return m.Success;
}

Where _reSmplExpr is defined like so:

private static Regex _reSmplExpr;
static ConcordPairEval()
{
  string pattern = @"({)(?<SmplExpr>([^{}])*?)(})";
  RegexOptions reo = RegexOptions.Singleline | RegexOptions.IgnoreCase;
  _reSmplExpr = new Regex(pattern, reo);
}

The pattern string above looks for a single open and a matching close brace with anything in between that contains no braces. Finding such a pattern will cause IsSimpleExpression(..) to return true.

Algorithmic limitation

The algorithm, as shown above, iteratively wades through the linked list, and evaluates a simple expression after simple expression. Now, when considering the possibilities of such an algorithm on the {if ..} construct, we see that it poses some limitations. So, {if ({=} = {=}) T, F} will not evaluate correctly as the algorithm evaluates simple expressions only.

Like the previous typographical limitations, these limitations are more of a theoretical concern than a practical concern. The limitations to the {if ..} construct are itemized in the first challenge I leave for you; see “Challenges for you”.

In closing

We met our motivations and goals:

• The algorithm can handle ConcordPair variables using the {key::name} construct, as well as {if ..} logic, and dynamic variables using the help of the ConcordPairOnTheFlyGlobal class.
• The algorithm reads the entire store and stuffs the entries into a data dictionary, allowing for entries known to the user only. Moreover, since the entries are in the dictionary, the entries known to the user only can be used within other entries.
• The {key::name} variable is case evaluated in a insensitive manner; moreover, most {key::body} variables have quite a bit that is case insensitive.
• ConcordPair variables can be nested within each other or specified side by side.
• {key::name} can reference forward or backward entries within the store.
• Having the DLL agnostic to the store type allows the operation on the variables to operate on any store type. The responsibility for reading the store and initializing the delegates are delegated to the client.
• The algorithm allows for other variables to be defined by providing other ProcessXxx classes—extendible.
• Finally, the framework is quite simple.

Conclusion

The ability to include variables that change dynamically in addition to static variables and if-logic constructs opens a slew of possibilities that were more difficult in the past and have just become easier.

An obvious application is one where you need to maintain different configuration files for different environments like Dev, UAT, and Prod. Now, you will be able to keep and maintain one file.

Another such place where these variables are indispensible is: batch processing, where there is a need for “user input” but no input screen can be provided.

Another instance where these variables are comfortable or indispensible is a place where the initialization is divided into two, an expert initialization and a user initialization. The expert initialization is done upfront. The expert initialization will use the variables described within the article.

Challenges for you

One issue that I left unresolved since article II, http://www.codeproject.com/KB/miscctrl/ConfigurationFile2.aspx, is an {if ..} construct. The {if (condition) tExp, fExp} format follows the rest of the evaluation, simple expressions are the only expressions evaluated. As such, the following if-logic variable {if (condition) {,}, {,,}} will be evaluated to {if (condition) ,,,,}, which is not the intended use, nor will it be evaluated. A second issue with the {if ..} construct is an “=” value in the condition; for example, {if ({=} ={=}) Life is Great, Life is Good}. Again, since simple expressions are the only ones evaluated, the {if ..} construct will not evaluate correctly. Your challenge: fix it so that the above if-logic works.

Another challenge: Figure out how to throw the EvaluateConcordPair method into an infinite loop that is protected by the PassThroughUpperLimit upper limit. If the infinite loop that you figured out is not protected by the PassThroughUpperLimit upper limit, then write to me and figure out how to fix it.

Challenge 3: ProcessDate, part of the repertoire of the ProcessXxx classes has a format specification that accepts lower case year, lower case day, but upper case month; this matches ProcessCurrentTime. There is no reason for ProcessDate to have a case sensitive format. Your challenge is to accept the format in any case.

Challenge 4: The {if ..} construct evaluates the following conditions as true/false: a DirectoryExists(..), a FileExists(..), and an exp1=exp2. Notably missing are the numeric logical comparisons (<, <=, =, > and >=), the ability to specify case sensitive equal, ability to provide partial path to DirectoryExists(..) and FileExists(..), and other more exotic comparisons like “partial path” is pointing to the same directory as “full path”. Or IsValidPartialPath(..) and IsValidFullPath(..) or IsValidPath(..). And so on and so forth. Your challenge, shall you decide to accept it, expand the {if ..} construct to your needs.

If you are about to take this challenge on, my recommendation is to provide one or more ProcessXxx classes that perform the desired behavior. So, for example, the numeric logic will be captured with a ProcessNumEval class like so: {NumEval::op-name::num1, num2}, where op-name will be the desired operation, it may be a logical operation or even a “plus” etc…

Challenge 5: The logic herein demands that the braces are balanced. Moreover, there is no way to specify an open or close brace that is not a part of the variable construct format. In the second article, http://www.codeproject.com/KB/miscctrl/ConfigurationFile2.aspx, I described the potential of specifying an open or close brace by allowing one to prefix the brace with a backslash. The problem with the solution published is that we may have a situation where we want to have a path like so: {key::path1}\{key::path2}. The second article will not handle this situation well. Your challenge is to come up with a solution.

Challenge 6: Use an aspect oriented solution to simplify the access to the ConcordPair properties.

Tools used

VS 2008 was used for the coding example. "Visual Paradigm for UML 7.0 Community Edition" is the tool used for diagramming the UML diagrams.

Enjoy!

History

• 9/12/2009: Initial submission.
• 9/15/2009: Fixed article (added the diagram).