Morestachio 4.0. An Evolving .NET TextEngine

Introduction

Morestachio is a C# Text formatting engine that allows you to feed a Text based Template and some Data in almost any form and Morestachio will generate a Text based on your template and your Data.

The Engine has evolved to support more and more features over the years and was finally released in version 3.0 with a whole bunch of new features, refactorings and bugfixes.

The essence of this article should be to help users who want to upgrade morestachio to version 3.0 but also to give an overall view of all available features morestachio has to offer.

• Github: Morestachio

Morestachio can be installed via NuGet:

Install-Package Morestachio

• Github Release 3.0.0
• Github Release 4.3.0.406

Key-Features

1. Separated Tokenizer, Parser and Execution engine to allow precompilation of Templates for fast execution
2. DocumentTree stores the result of an parser to allow manipulation and serialization of an PreCompiled template
3. Stream based execution allows writing direct to Disk for very large templates
4. Limited Stream size for maximum control over generated templates in a "User creates template" case
5. Can process any kind of object as Data (like objects, IDictionary<string,object> and any IEnumerable
6. Variable Encoding
7. In template Data formatting
8. Template Partials for reusing templates
9. In Template Variable support
10. Data Scoping for easy readable Templates

Morestachio Playground

As morestachio is compliant with .NET Standard 2.0, it can be compiled to .NetCore 3.0 and therefore can be used in Blazor Webassembly. You can take advantage of that fact by using the Morestachio Playground Editor at: https://morestachio.jean-pierre-bachmann.dev/.

This Editor runs 100% in your browser and allows you to play with morestachio.

Basic Template Processing

First, you have to understand how Morestachio should be used. Morestachio is divided into 3 different structures, the Tokenizer, the Parser and the DocumentTree.

Tokenizer

The Tokenzier will (like the name implicates) first Tokenizes your Template. That means it runs from Top to bottom and will search for anything within "{{" "}}", it will ensure that the templates do not contain invalid syntax and will provide a 1Dimensional array of TokenPairs. After the Tokenzier has run, your template will no longer be viewed by the engine. After the Tokenzier has run, the result is a list of TokenPairs and a list of Errors. The List of TokenPairs are not very intuitive to use or to manipulate and your Template is validated but not evaluated.

Quote:

Starting with version 4.3.0.397 morestachio introduced the {{#SET OPTION }} keyword that allows some control over certain tokenizer properties from within the template itself. For Exmaple you can change the delimiter tokens from {{ and }} to anything else like {{#SET OPTION TokenPrefix = "<%"}}

Parser

The Parser will interpret the result of the Tokenizer and compiles a DocumentTree. This Tree contains specific instances of IDocumentItem and have the methods to execute the desired behavior. You can serilize the DocumentTree to Json, Binary or XML as they all have the corresponding interfaces. The execution of the Parser is internally always tied to the execution of the Tokenzier and both will be executed by calling Parser.ParseWithOptions(ParserOptions):MorestachioDocumentInfo. To get a result, you have to call Create on the MorestachioDocumentInfo you obtain from the ParseWithOptions method.

Parser Options

You can control certain aspects of the generation with the ParserOptions class.

ParserOptions

Template :string	The string template you want to process
Timeout :TimeSpan	A maximum execution time after the execution should be cancelled
CustomDocumentItemProviders: IList<CustomDocumentItemProvider>	A list of custom template keywords
ValueResolver: IValueResolver	A Value Resolver that can be used to extend the Build-in resolver
UnresolvedPath: event	An event that will be triggered to log a Path that is not available in the Data
Formatters: IMorestachioFormatterService	The service that provides access to data formatters
PartialStackSize: uint	The maximum depth of Partials
StackOverflowBehavior: PartialStackOverflowBehavior	How a Partial that exceeds the PartialStackSize should be handled
DisableContentEscaping: bool	Should all values be HTML escaped
MaxSize: long	The maximum size in bytes that the Template should not be exceed. Accurate up to 1 byte
SourceFactory: Func<Stream>	The Factory for the generation of a stream that should be the target of the Template generation.
Encoding	The Encoding that should be used to write to the Stream
Null: string	How should null values be written into the Template

Upgrade Guide from 2.0 to 3.0

A lot has changed in the time from version 2.0 to 3.0. A lot of new features were added and the whole Formatter framework was rewritten from scratch. This also introduced a breaking change in how to invoke a formatter. Also, Morestachio was rewritten as a .NetStandard lib and currently supports:

• netstandard2.0;
• netcoreapp2.0;
• netcoreapp2.1;
• netcoreapp2.2;
• netcoreapp3.0;
• net46;
• net461;
• net462;
• net47;
• net471;
• net472;

Partials

Morestachio allows Partials to be added anywhere in the Template with the #Declare NAME and #Include NAME syntax. Partials can be recursive and are always tied to the same scope as the invoker.

Example:

{{#DECLARE CodeProjectPartialName}}
I am static text that will be included into the include call
{{Data.Test}}
{{/DECLARE}}

{{#INCLUDE CodeProjectPartialName}}

To include external Partials, I recommend using the ParserOptions.PartialStore. This store allows adding parsed templates or you can overwrite the IPartialStore interface to provide your own Partial logic.

Boxed Objects

Apart from enumerating an IDictionary<string, object>, you can now iterate over it with the .? suffix in a path. This can even be used to iterate objects. A Path that ends with .? will always yield a list of {Key: string, Value: object}s.

Example:

{{#EACH Data.Object.?}} <- it does not matter if the value of "Data.Object" 
is an C# object or an IDictionary<string,object>
{{Key}} <- prints the name of a property of Data.Object
  {{Value}} <- prints the value of the property of Data.Object
{{/EACH}}

Non Scoping IF & IfNot

Different from the Scope keyword #Data and ^Data, you can just use an #IF Data and ^IF Data to check for existence without scoping to that object.

Fluent Formatters

A breaking change has been introduced with version 3.0. All formatters should now be written as part of a path as opposed in version 2.0 as the first argument of an call.

Old syntax:

{{Data("formatterName", argumentPathA, "constStringArgument2")}}

This has changed to:

{{Data.formatterName(argumentPathA, "constStringArgument2")}}

This adds more readability and is overall cleaner to write.

To Invoke a build in formatter or call a formatter that does not specify any name, you just have to leave the formatter name like this:

{{Data.()}}

in the next version, the ToString formatter will be added as an alias for the default formatting.

.NET Native Type Support

Morestachio allows 4 Native C# types within the template:

1. Numbers {{123.3}} {{123D}} {{123L}} {{123UL}} {{123}}
2. Boolean {{true}} {{false}}
3. Strings {{"str"}} {{'str'}}
4. nulls {{null}}

All these types can also be used as an argument for an formatter. Numbers, strings and boolean can be used as the source for an formatter, only null cannot.

1. Numbers
   1. {{123.3.Add(123)}} {{123D.Add(123)}} {{123L.Add(123L)}} {{123UL.Add(123UL)}} {{123.Add(123)}}
2. Boolean {{true.IsTrue(true)}} {{false.IsTrue(false)}}
3. Strings {{"str".Replace("t", "d")}} {{'str'.Replace("t", "d")}}
4. nulls {{Data.IsEquals(null)}}

Aliases and Variables

You can define custom variables and set them to any value you like and use Alias in certain keywords to define a variable for the result of an expression.

Variables and aliases are stored the same so if you define a Alias, you can overwrite it with any value you like.

You can define an alias for SCOPE (#), EACH and IFs.

Example:

{{#EACH Data.Values}} <-no alias you are always scoped to the value
{{PropA}} <- accesses the nth element in Values
{{/EACH}}

{{#EACH Data.Values AS item}} <-access the nth value of Data.Values over item
{{item.PropA}} <- accesses item
{{/EACH}}

Quote:

Currently, the Alias for a collection item is not a substitute for a scope, but it's likely that will change in a future version. I recommend using an alias for every #EACH.

Variables can be created by using the #VAR NAME = expression.

Example:

{{#var item = Data.PropertyA}} <- creates or overwrites a variable named "item" 
  with the value of Data.PropertyA
{{item.PropertyV}} <- Prints the property of the value in variable "item"

{{#var item = Data.PropertyZ}} <- overwrites the variable item 
                                  with the value of Data.PropertyZ

{{#EACH Data.Values AS item}} <- overwrites the variable item with the 
                                 value of Data.Values[index]
    {{item}} <- Prints the nth item in Data.Values
    {{#var item = Data.PropertyZ}} <- overwrites the variable item 
                                      with the value of Data.PropertyZ
{{/EACH}}

Document Tree

The biggest code wise change was made in the DocumentTree. All Keywords and Functions have now their corresponding IDocumentItem. They are nested to allow a structured tree of responsibilities. The whole tree can be traversed down and allow (partial) modifications to enable post-processing of templates but the whole tree is also serializable. This feature allows the Precompilation and storage of that result to enable very fast rendering of a template multiple times. It is also the first step onto a full Editor.

Example of serialized Template:

Assume this Template:

I am <Text> {{Data.data.test().next(arg).(last)}}

For an XML serialization, this will result in an XML document like this:

<?xml version="1.0"?>
<MorestachioDocument Kind="Document" MorestachioVersion="3.0.0.0">
  <Children>
    <ContentDocumentItem Kind="Content" ExpressionStart="1:1">
      <Value xml:space="preserve">I am &lt;Text&gt; </Value>
    </ContentDocumentItem>
    <IsolatedContextDocumentItem Kind="IsolatedContext" ExpressionStart="1:13">
      <Children>
        <CallFormatterDocumentItem Kind="CallFormatter">
          <Value xml:space="preserve">Data.data</Value>
          <TargetFormatterName>test</TargetFormatterName>
        </CallFormatterDocumentItem>
        <IsolatedContextDocumentItem Kind="IsolatedContext" ExpressionStart="1:29">
          <Children>
            <CallFormatterDocumentItem Kind="CallFormatter">
              <TargetFormatterName>next</TargetFormatterName>
              <FormatString>
                <Argument>
                  <PathDocumentItem Kind="Expression" EscapeValue="False">
                    <Value xml:space="preserve">arg</Value>
                  </PathDocumentItem>
                </Argument>
              </FormatString>
            </CallFormatterDocumentItem>
            <IsolatedContextDocumentItem Kind="IsolatedContext" ExpressionStart="1:39">
              <Children>
                <CallFormatterDocumentItem Kind="CallFormatter">
                  <TargetFormatterName />
                  <FormatString>
                    <Argument>
                      <PathDocumentItem Kind="Expression" EscapeValue="False">
                        <Value xml:space="preserve">last</Value>
                      </PathDocumentItem>
                    </Argument>
                  </FormatString>
                </CallFormatterDocumentItem>
                <PrintContextValue Kind="PrintExpression" />
              </Children>
            </IsolatedContextDocumentItem>
          </Children>
        </IsolatedContextDocumentItem>
      </Children>
    </IsolatedContextDocumentItem>
  </Children>
</MorestachioDocument>

Finally

I want to thank you very much for reading the whole document. I have been developing this project for more than 1 1/2 years in my freetime now and I would love to hear your impressions and or tips for features. I maintain this project alone and will respond on Github to each ticket.

History

• 9^th March, 2020: Initial version
• 25^th September, 2020: Added info about Playground editor