Click here to Skip to main content
Click here to Skip to main content

HOPE - Higher Order Programming Environment

, 10 Jul 2014
Rate this:
Please Sign up or sign in to vote.
A Dynamic Runtime Semantic Computation Environment

Source code is located on GitHub: https://github.com/cliftonm/HOPE

Watch the movie!  (Also see the github page for additional video links)

Article #2: Scraping the Astronomy Picture Of the Day website, can be found here.

Article #3: Hunt the Wumpus in HOPE, can be found here.

Introduction

As I write this, tomorrow will be the 12 year anniversary of my first article on the Code Project: the Organic Programming Environment (OPEN). The concept, an open surface on which one places data that then triggers processes (also runtime introduced), is one that I revisit occasionally. Originally written in C++, OPEN did not have the advantages of:

  • reflection
  • runtime compilation
  • and dynamic typing

that are available today in many languages. These are the cornerstones that bring OPEN into the today's world, re-implemented as HOPE in C#. The salient features are:

  • "Receptor" components (assemblies) that can be dropped onto the surface
  • Semantic meaning associated with data
  • "Carriers" representing the data "protocol" and the "signal" (data and semantic meaning)
  • A visualization system animates the interaction (via carriers) between receptors

So what is HOPE? It is a dynamic runtime semantic computation environment.

Computational Semantics

What HOPE implements is a conceptual space (see Appendix B) for the purposes of "automating the process of constructing and reasoning with meaning representations." Furthermore, "the meaning of individual units can combine to form the meaning of larger units."

Demonstrations

Three demonstrations are provided in this release:

  1. A simple "Hello World" heartbeat
  2. An images-to-thumbnails converter and viewer
  3. "Today's weather", which, using .NET's speech synthesizer, speaks the current weather conditions.

Credits

My interest in resurrecting OPEN and in creating the HOPE system is due to my friends Eric Harris-Braun and Arthur Brock. Eric and Arthur (and others) are working on a project (Metacurrency) whose architecture (Ceptr) is the inspiration for HOPE. One could say that they are working on the low-level architecture while HOPE is a high-level prototype demonstrating their concepts. Of course, one of the things that attracted me immediately to their project was that, at its core, it implements the concepts that I introduced with OPEN. As I'm also involved in their project as a "documenter", there may be other things from that project that end up here on Code Project, and certainly elsewhere.

Terminology

The terminology that I've already introduced is borrowed from Eric's and Arthur's "Ceptr" project ("a compute platform for a decentralized self-describing protocol stack.") I don't profess a direct mapping of their terminology with the concepts here, but they are similar enough at a high level that I feel that anyone reading an article on Ceptr would be able to immediately understand the use of the same terms here. And vice-versa, since this is actually the first article on the architecture concepts, I am hoping that this article will be a segue for introducing terms that you will encounter in later articles.

Here are the terms and their definitions that I'll be using:

Receptor System

The assembly responsible for managing the collection of receptors (see next) and messaging between them.

Receptor

A .NET assembly that has a class implementing IReceptorInstance, allowing it to register itself with the receptor system, stating what carriers (see next) it is interested in receiving. When a carrier is received, the receptor activates a work process (on the same thread or on a separate thread) that results either in an internal state change, one or more carriers being emitted, or some "edge" process, or some combination of the above.

Edge Receptor

A receptor that interfaces with the outside world -- that is, the rest of the computer platform and the operator (through audio-visual means).

Protocol

The definition of a set (or stream) of data. The protocol determines what data is where. In a stream lacking semantic information, the protocol provides a means for parsing the stream into semantic data--data that has meaning. In this prototype, the protocol is determined declaratively (in XML) and is fully semantic.

Signal

While the protocol is a declaration of how to parse the values, the signal is the actual values.

Carrier

The messaging construct between receptors. A carrier is implemented as an object consisting of a protocol and a signal (see next), such that the signal can be mapped to "meaning." XML is itself a good example of a carrier--it incorporates both protocol and signal. Elements have meaningful names and attribute-value pairs specify the meaning of the value. Furthermore, with XSD, one can further specify the semantics of an element or attribute, describing relationship, ordinality, and so forth.

Visualization

Visualizing the data, processes, and flows as the system performs its computations is not only aesthetically pleasing but it also provides a concrete interface for which to interact with the system.

  • Receptors (assemblies) can be dragged from Explorer and dropped into the surface
  • Pre-canned carriers can also be dragged and dropped onto the surface
  • Carriers that exist on the surface but have no receptors (at the moment) are easily visualized
  • Additional rendering can be performed to facilitate the user experience.

Because of the visual nature of HOPE, there will be links to movies illustrating what, even in its nascent state, it is capable of.

Receptors

We'll do a brief walk-through of receptors.

Boilerplate

Every receptor is required to implement IReceptorInstance. The boilerplate for a receptor looks like this:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.Windows.Forms;

using Clifton.Receptor.Interfaces;
using Clifton.SemanticTypeSystem.Interfaces;

namespace AReceptor   // Name this as you wish
{
  // The class name is completely arbitrary
  public class ReceptorDefinition : IReceptorInstance
  {
    // The Name property is used by the visualizer.
    public string Name { get { return "Heartbeat"; } }

    // Also used by the visualizer to distinguish edge receptors.
    public bool IsEdgeReceptor { get { return false; } }

    // Prevents the receptor from being rendered at all.
    public bool IsHidden { get { return false; } }

    // Your interface to the receptor system.
    protected IReceptorSystem rsys;

    // Constructor.  You will be passed the IReceptorSystem instance.
    public ReceptorDefinition(IReceptorSystem rsys)
    {
      this.rsys = rsys;
      // Do your receptor-specific initialization here.
    }

    // Called when the receptor is removed or the application is closed.
    public void Terminate()
    {
      // Stop any worker threads, dispose unmanaged objects, etc.
    }

    // Returns a list of carrier names that this receptor wants to monitor.
    public string[] GetReceiveCarriers()
    {
      return new string[] {};
    }

    // Processes a carrier.
    public void ProcessCarrier(ISemanticTypeStruct protocol, dynamic signal)
    {
    }
  }
}

Things To Improve

In the implementation, as it stands right now, carrier names are literal strings (bad) and need to be checked if the receptor handles more than one carrier (also bad.)

Also, the protocols are stored in a separate XML file. Ideally, a receptor that emits carriers should define the protocols.

The Heartbeat Receptor -- An Example

Here we have a "Heartbeat" receptor. Given the boilerplate above, we add:

public ReceptorDefinition(IReceptorSystem rsys)
{
  this.rsys = rsys;
  InitializeRepeatedHelloEvent();
}

public void Terminate()
{
  timer.Stop();
  timer.Dispose();
}

protected void InitializeRepeatedHelloEvent()
{
  timer = new Timer();
  timer.Interval = 1000 * 2; // every 10 seconds.
  timer.Tick += SayHello;
  timer.Start();
}

protected void SayHello(object sender, EventArgs args)
{
  ISemanticTypeStruct protocol = rsys.SemanticTypeSystem.GetSemanticTypeStruct("DebugMessage");
  dynamic signal = rsys.SemanticTypeSystem.Create("DebugMessage");
  signal.Message = "Hello World!";
  rsys.CreateCarrier(this, protocol, signal);
}

This receptor does not listen for any carriers, it simply emits a carrier every two seconds.

The Logger Receptor

The Logger receptor, however, listens to carriers containing the "DebugMessage" protocol:

public string[] GetReceiveCarriers()
{
  return new string[] { "DebugMessage" };
}

public void ProcessCarrier(ISemanticTypeStruct protocol, dynamic signal)
{
  string msg = signal.Message;
  System.Diagnostics.Debug.WriteLine(msg);

  Flyout(msg);
}

protected void Flyout(string msg)
{
  ISemanticTypeStruct protocol = rsys.SemanticTypeSystem.GetSemanticTypeStruct("SystemMessage");
  dynamic signal = rsys.SemanticTypeSystem.Create("SystemMessage");
  signal.Action = "Flyout";
  signal.Data = msg;
  signal.Source = this;
  rsys.CreateCarrier(this, protocol, signal);
}

This receptor does do things -- it outputs to the debug stream the signal content (the message in this case) as well as creating a carrier instructing the application to create a flyout animation of the message.

Behind the Scenes

Notice that the signal is of type dynamic. In order to support the tenet of HOPE, "a dynamic runtime semantic computation environment", protocols cannot (usually) be hard-coded. We do not require foreknowledge of a carrier's protocol nor do receptors necessarily care whether the protocol is ever extended. What the dynamic keyword lets us do is write, in shorthand, the code necessary to reflect on the signal knowing the protocol. Yes, this means we're doing duck-typing with carrier signals. Some discussion of the semantic type system is described in Appendix C.

Carriers

Carriers are trivial. They describe the protocol and the signal instance:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;

using Clifton.Receptor.Interfaces;
using Clifton.SemanticTypeSystem.Interfaces;

namespace Clifton.Receptor
{
  public class Carrier : ICarrier
  {
    public ISemanticTypeStruct Protocol { get; set; }
    public dynamic Signal { get; set; }

    public Carrier(ISemanticTypeStruct protocol, dynamic signal)
    {
      Protocol = protocol;
      Signal = signal;
    }
  }
}

Note how the protocol is a semantic type.

Receptor System

The receptor system manages receptors and carrier creation. The current implementation is designed to work with the visualizer (indirectly, of course), such that the action to perform when a carrier is paired with a receptor is handed off to the visualizer to invoke when the carrier reaches its destination by whatever mechanism the visualizer uses to animate the carrier:

protected Action GetProcessAction(IReceptorInstance from, Carrier carrier, bool stopRecursion)
{
  Action action = new Action(() =>
  {
    var receptors = CarrierReceptorMap[carrier.Protocol.DeclTypeName];
    int numReceptors = receptors.Count;

    receptors.ForEach(receptor =>
    {
      Action process = new Action(() => receptor.Instance.ProcessCarrier(carrier.Protocol, carrier.Signal));

      if (receptor.Instance.IsHidden)
      {
        // Don't visualize carriers to hidden receptors.
        process();
      }
      else if (!stopRecursion)
      {
        ISemanticTypeStruct protocol = SemanticTypeSystem.GetSemanticTypeStruct("CarrierAnimation");
        dynamic signal = SemanticTypeSystem.Create("CarrierAnimation");
        signal.Process = process;
        signal.From = from;
        signal.To = receptor.Instance;
        signal.Carrier = carrier;
        CreateCarrier(null, protocol, signal, receptor.Instance.GetReceiveCarriers().Contains("*"));
      }
    });
  });

  return action;
}

Notice how when the receptor is hidden, the action is invoked immediately, since there is no visualization. Also notice how a carrier is created which the application receives (the application is itself a receptor!) for animating the carrier. The power of lambda expressions and closures!

The stopRecursion is required to stop wildcard receivers from receiving their own messages ad-infinitum.

At some point, this will all be refactored so that the receptor system won't even know whether there is a visualizer that is injecting itself into the process.

Queuing Carriers

Another aspect of this system is that carriers are accumulated if there are no receptors. This can be good or bad, depending on the desired process, and currently there is no mechanism to indicate whether a newly created carrier should replace an existing one or not. The above animation illustrates a receptor that has accumulated unprocessed carriers and what happens when we drop a receptor (an assembly) onto the surface that can process those carriers.

protected void ProcessReceptors(IReceptorInstance from, Carrier carrier, bool stopRecursion)
{
  Action action = GetProcessAction(from, carrier, stopRecursion);
  List<Receptor> receptors;

  bool haveCarrierMap = CarrierReceptorMap.TryGetValue(carrier.Protocol.DeclTypeName, out receptors);
 
  // If we have receptors for this carrier (a mapping of carrier to receptor list exists and receptors actually exist)...
  if (haveCarrierMap && receptors.Count > 0)
  {
    // Perform the action.
    action();
  }
  else
  {
    // Othwerise, queue up the carrier for when there is a receptor for it.
    queuedCarriers.Add(new CarrierAction() { Carrier = carrier, Action = action });
  }
}

Here we're queuing the action at the time the carrier is created if there are no receptors interested in receiving a carrier with the specified protocol.

One of the interesting points about all this is, when the carrier is created:

public void CreateCarrier(IReceptorInstance from, 
                          ISemanticTypeStruct protocol, 
                          dynamic signal, 
                          bool stopRecursion = false)
{
  Carrier carrier = new Carrier(protocol, signal);
  NewCarrier.Fire(this, new NewCarrierEventArgs(from, carrier));

  // We stop recursion when we get to a wild-card carrier receptor, otherwise, 
  // the "Carrier Animation" message recursively creates additional messages to the wild-card receptor.
  ProcessReceptors(from, carrier, stopRecursion);
}

It is created locally, but through the mechanism of closures, we can later process that instance. Also note that the visualizer hooks the NewCarrier event so that it can show these carriers accumulating around the receptor that is generating them.

Image Thumbnail Creator and Viewer Example

There are three receptors provided that facilitate creating thumbnail images of JPG's dropped onto the surface:

  1. ImageWriterReceptor - writes the thumbnail to the same location with the text "-thumbnail" appended to the filename.
  2. ThumbnailCreatorReceptor - receives carriers with a JPG image protocol and converts them to smaller images
  3. ThumbnailViewerReceptor - reparses the carrier output of the thumbnail creator into a carrier that the visualizer (part of the application) handles.

What you see here are three images being dropped onto the surface. These are packaged as carriers and received by the thumbnail creator. There are two receptors interested in thumbnail images, the viewer and the file writer, and we see the carriers for the thumbnail pictures vectoring off to the receptors of those carriers. The thumbnail viewer shows the images in a radius around the viewer receptor. One thing to note here is that the carrier is not being cloned, therefore processes that operate on the carrier's signal should be cognizant that other processes might be operating on them simultaneously. For example, because an image is not thread safe, you cannot render it in GDI+ and save it to disk at the same time. The whole issue of multi-threading becomes very interesting indeed in this kind of architecture.

Dragging and Dropping

Image files dropped onto the surface create carriers with a specific protocol:

// Create carriers for each of our images.
ISemanticTypeStruct protocol = Program.SemanticTypeSystem.GetSemanticTypeStruct("ImageFilename");
dynamic signal = Program.SemanticTypeSystem.Create("ImageFilename");
signal.Filename = fn;
// TODO: The null here is really the "System" receptor.
Program.Receptors.CreateCarrier(null, protocol, signal);

Thumbnail Creator Receptor

This receptor processes the image and creates a carrier with a thumbnail protocol:

public string[] GetReceiveCarriers()
{
  return new string[] { "ImageFilename" };
}

public async void ProcessCarrier(ISemanticTypeStruct protocol, dynamic signal)
{
  string fn = signal.Filename;
  Image ret = await Task.Run<Image>(() =>
  {
    Bitmap bitmap = new Bitmap(fn);
    // Reduce the size of the image. If we don't do this, scrolling and rendering of scaled images is horrifically slow.
    Image image = new Bitmap(bitmap, 256, 256 * bitmap.Height / bitmap.Width);
    bitmap.Dispose();

    return image;
  });

  OutputImage(fn, ret);
}

protected void OutputImage(string filename, Image image)
{
  ISemanticTypeStruct protocol = rsys.SemanticTypeSystem.GetSemanticTypeStruct("ThumbnailImage");
  dynamic signal = rsys.SemanticTypeSystem.Create("ThumbnailImage");
  // Insert "-thumbnail" into the filename.
  signal.Filename = filename.LeftOfRightmostOf('.') + "-thumbnail." + filename.RightOfRightmostOf('.');
  signal.Image = image;
  rsys.CreateCarrier(this, protocol, signal);
}

Thumbnail Viewer Receptor

This receptor hands off the image to the system's visualizer. All it does is revector the carrier, somewhat like an intermediate neuron.

public string[] GetReceiveCarriers()
{
  return new string[] { "ThumbnailImage" };
}

public void ProcessCarrier(ISemanticTypeStruct protocol, dynamic signal)
{
  Image image = signal.Image;
  ShowImage(image);
}

protected void ShowImage(Image image)
{
  // All we do here is send a message to the system so that it displays our thumbnails as a carousel around or receptor.
  ISemanticTypeStruct protocol = rsys.SemanticTypeSystem.GetSemanticTypeStruct("SystemShowImage");
  dynamic signal = rsys.SemanticTypeSystem.Create("SystemShowImage");
  signal.From = this;
  signal.Image = image;
  rsys.CreateCarrier(this, protocol, signal);
}

Thumbnail Writer Receptor

public string[] GetReceiveCarriers()
{
  return new string[] { "ThumbnailImage" };
}

public void ProcessCarrier(ISemanticTypeStruct protocol, dynamic signal)
{
  string fn = signal.Filename;
  Image img = signal.Image;

  img.Save(fn);
}

Today's Weather Example

Here we have several receptors. Because the web service calls take a while, I haven't created an animated GIF. Furthermore, since the weather is spoken, there's not much to "see" except for carriers flying about.

There are two salient points though. One, we can drop a carrier, as an XML fragment, on to the surface:

<Carriers>
  <Carrier Protocol="Zipcode" Value="12565"/>
</Carriers>

This is a useful way of injecting signals into the system. The zipcode is received by both the weather service and zip code service receptors, which invoke web services to acquire their information. For example, the zipcode service processes the zipcode signal and emits a carrier with a Location protocol and signal:

public async void ProcessCarrier(ISemanticTypeStruct protocol, dynamic signal)
{
  Tuple<string, string> location = await Task.Run(() =>
  {
    string city = String.Empty;
    string stateAbbr = String.Empty;

    try
    {
      string zipcode = signal.Value;
      USZip zip = new USZip();
      XmlNode node = zip.GetInfoByZIP(zipcode);
      XDocument zxdoc = XDocument.Parse(node.InnerXml);
      city = zxdoc.Element("Table").Element("CITY").Value;
      stateAbbr = zxdoc.Element("Table").Element("STATE").Value;
    }
    catch (Exception ex)
    {
      // TODO: Log exception.
      // Occasionally this web service will crash.
    }

    return new Tuple<string, string>(city, stateAbbr);
  });

  Emit(signal.Value, location.Item1, location.Item2);
}

protected void Emit(string zipCode, string city, string stateAbbr)
{
  ISemanticTypeStruct protocol = rsys.SemanticTypeSystem.GetSemanticTypeStruct("Location");
  dynamic signal = rsys.SemanticTypeSystem.Create("Location");
  signal.Zipcode = zipCode;
  signal.City = city;
  signal.State = "";
  int idx = Array.IndexOf(abbreviations, stateAbbr);

  if (idx != -1)
  {
    signal.State = abbreviations[idx + 1];
  }

  signal.StateAbbr = stateAbbr;
  rsys.CreateCarrier(this, protocol, signal);
}

Notice how the webservice is invoked asynchronously with the Task.Run method and upon completion, the Location protocol and signal is packaged up into a carrier.

The Weather Info receptor is receiving carriers with Location and WeatherInfo protocols, and when the zipcodes match, it formats the data into a way that can be spoken and sends off the text strings as carriers that the TextToSpeech receptor is listening for. The TextToSpeech receptor is straightforward:

public class ReceptorDefinition : IReceptorInstance
{
  public string Name { get { return "Text To Speech"; } }
  public bool IsEdgeReceptor { get { return true; } }
  public bool IsHidden { get { return false; } }

  protected SpeechSynthesizer speechSynth;
  protected Queue<string> speechQueue;
  protected IReceptorSystem rsys;

  public ReceptorDefinition(IReceptorSystem rsys)
  {
    this.rsys = rsys;

    speechSynth = new SpeechSynthesizer();
    speechSynth.SpeakCompleted += OnSpeakCompleted;
    speechSynth.Rate = -4;
    speechQueue = new Queue<string>();
  }

  public string[] GetReceiveCarriers()
  {
    return new string[] { "TextToSpeech" };
  }

  public void Terminate()
  {
  }

  public void ProcessCarrier(ISemanticTypeStruct protocol, dynamic signal)
  {
    string msg = signal.Text;
    Speak(msg);
  }

  protected void OnSpeakCompleted(object sender, SpeakCompletedEventArgs e)
  {
    if (speechQueue.Count > 0)
    {
      string msg = speechQueue.Dequeue();
      speechSynth.SpeakAsync(msg);
    }
  }

  protected void Speak(string msg)
  {
    if (speechSynth.State == SynthesizerState.Speaking)
    {
      speechQueue.Enqueue(msg);
    }
    else
    {
      speechSynth.SpeakAsync(msg);
    }
  }
}

The Carrier Exporter

This is a "listen to everything" receptor, and it's purpose is to log all the carriers as an XML stream.

public string[] GetReceiveCarriers()
{
  return new string[] { "*" };
}

public void Terminate()
{
  XmlWriterSettings xws = new XmlWriterSettings();
  xws.Indent = true;
  xws.OmitXmlDeclaration = true;
  XmlWriter xw = XmlWriter.Create("carrier_output.xml", xws);
  xdoc.WriteTo(xw);
  xw.Close();
}

public void ProcessCarrier(ISemanticTypeStruct protocol, dynamic signal)
{
  XmlNode carrier = xdoc.CreateElement("Carrier");
  carrier.Attributes.Append(CreateAttribute(xdoc, "Protocol", protocol.DeclTypeName));
  carriersNode.AppendChild(carrier);

  Type t = signal.GetType();
  t.GetProperties(BindingFlags.Instance | BindingFlags.Public).ForEach(p =>
  {
      ... etc ....

When this receptor is on the surface, you'll notice that all your carriers are sent to both the exporter receptor and any other receptors interested in the carrier's protocol. So, for example, here is the run for dropping the zip code for where I live onto the surface:

<Carriers>
  <Carrier Protocol="TextToSpeech" Text="Loading receptors." />
  <Carrier Protocol="TextToSpeech" Text="Receptor System online." />
  <Carrier Protocol="TextToSpeech" Text="Loading receptors." />
  <Carrier Protocol="TextToSpeech" Text="Receptors online." />
  <Carrier Protocol="TextToSpeech" Text="Processing carriers." />
  <Carrier Protocol="Zipcode" Value="12565" />
  <Carrier Protocol="Location" Zipcode="12565" City="Philmont" StateAbbr="NY" State="NEW YORK" />
  <Carrier Protocol="WeatherInfo" Zipcode="12565" Summary="Isolated Thunderstorms" High="75" Low="51">
    <Conditions>
      <WeatherCondition Coverage="isolated" Intensity="none" WeatherType="thunderstorms" Qualifier="gusty winds,small hail" />
      <WeatherCondition Additive="and" Coverage="isolated" Intensity="light" WeatherType="rain showers" Qualifier="none" />
    </Conditions>
    <Hazards />
  </Carrier>
  <Carrier Protocol="TextToSpeech" Text="Here is the weather for Philmont NEW YORK. " />
  <Carrier Protocol="TextToSpeech" Text="The low is 51." />
  <Carrier Protocol="TextToSpeech" Text="The high is 75." />
  <Carrier Protocol="TextToSpeech" Text="The weather is: Isolated Thunderstorms." />
  <Carrier Protocol="TextToSpeech" Text="Today there will be isolated thunderstorms gusty winds, small hail and isolated light rain showers." />
  <Carrier Protocol="TextToSpeech" Text="There are no hazards in effect." />
</Carriers>

Notice we completely exclude from and to information -- that is actually irrelevant. What we're interested in is the protocol and signal, packaged by a carrier.

Purpose of the Carrier Exporter Receptor

In addition to logging the activity of the system, this is a great debug tool. One can easily extract specific carrier-protocol-signal instances and test the system by dropping that carrier onto the surface. So, instead of, say, dropping a zipcode every single time onto the surface to test the parsing of the weather data into speech, we can instead drop the carrier-protocol-signal from a specific run and re-use that carrier over and over.

Conclusion

The neat thing about this system is actually how dynamic it is -- how easy it is to write receptors and behaviors in "lower-order" programming style and integrate them to accomplish new behaviors. Re-usability is not just "link to the library" but actually "drop this package onto the surface and let it start interacting with the system." This provides a richer, code-free context in which to support an unlimited number of interesting behaviors.

There are huge number of details that still need to be worked out. By mixing lower and higher order programming styles, we can implement behaviors as small, re-usable components. It should be possible to write open applications, one that can be quickly customized and enhanced. That is my HOPE, haha.

HOPE will return...

Appendix A

For example, I recently encountered this Ruby code (greatly simplified) in a commercial website:

def update_value(value_name, new_value)
  persist_change(value_name, new_value, @value)
  @value = new_value
end

def persist_change(name, old, new)
  ...
end

Notice how the the new and old parameters are reversed. Now, in a duck-typing language like Ruby, you are somewhat hard-pressed to get the runtime to fault on this. At best, you could create a couple class with specific methods that would result in a runtime fault:

class OldValue
  property_accessor :old_value
end

class NewValue
  property_accessor :new_value
end

Here, the semantic meaning must be implemented as unique property accessor methods.

In strongly typed languages such as C#, we can fault at compile time because the semantic declaration of the class itself is different:

public class OldValue<T>
{
  public T Value {get;set;}
}

public class NewValue<T>
{
  public T Value {get;set;}
}

Now, if in C# we write:

void UpdateValue(string name, NewValue<string> val)
{
  PersistChange(name, val, new OldValue<string>() {Value = myName});
  myName = val.Value;  // assuming the "name" we're updating is "name"
}

void PersistChange(string name, OldValue<string, NewValue<string>)
{
  ...
}

This code will fail to compile because we have provided enough semantic meaning such that the compiler can distinguish between old and new string values. In F#, the semantics of a value can be more easily expressed using discriminated unions. The Julia Language type system is even more supportive of attaching semantics to value.

Appendix B

A critical requirement in OPEN's is the incorporation of semantic meaning associated with the data. One cannot just drop the number "42" onto the surface without having an attached semantic meaning. Computation is not possible without semantic meaning. For example, unless we know what "42" means, we cannot perform computations with it. Even today, there are two problems with modern computation:

  1. The meaning of a value is not a part of the value.
  2. Any associated meaning is decoupled from the value.

For a simple example, see Appendix A.

As a result, I postulate that our computational capacities are severely crippled by the fact that our data does not carry its semantics with it. In other words, without knowing about the meta-world of our data, we cannot write processes that perform computations on that data automatically when it is presented into a system. As long as we continue to generate reams of data in our databases, in our blogs, on our websites that have no semantic meaning attached, we are dealing essentially with dead and useless "information." Semantics converts this data into useable "knowledge." That is what HOPE demonstrates (hopefully!)

Formal Semantics

From "Formal Semantics" by Julia Hockenmaier, Laura McGarrity, Bill McCartney, Chris Manning, and Dan Klein:

Formal semantics consists of:

  • Lexical Semantics: the meaning of words
  • Compositional Semantics: how the meaning of individual units combine to form the meaning of larger units

Meaning is not specifically a dictionary entry but rather an association of semantics. In the link above, the word "rooster" is given meaning by the semantic association with the words "male" and "chicken." In compositional semantics, "The meaning of a sentence is determined by the meaning of its words in conjunction with the way they are syntactically combined." In terms of computer data, it is often how the data is organized along with words (such as "get" or "put") that determine the meaning, for example, of an HTTP message.

Cognitive Semantics

From Cognitive Semantics: "Cognitive semantics holds that language is part of a more general human cognitive ability, can therefore only describe the world as it is organized within people's conceptual spaces. It is implicit that there is some difference between this conceptual world and the real world."

Computational Semantics

From Computaitonal Semantics: "Computational semantics is the study of how to automate the process of constructing and reasoning with meaning representations of natural language expressions." While HOPE is not about natural language expressions, it is about constructing and reasoning with meaning representations as represented by small process components and semantic data.

Appendix C

The intention of the semantic type system is to fully describe types, such that all types are derived from language-specific primitives (though technically they could be derived from actual bits) and that higher-order types are composed from lower-order types. I have a separate article (to be published) that discusses the semantic type system in detail. Here is a brief explanation. Note that he current HOPE protocols violate the pure concepts to some degree.

Here is the fundamental type, "Noun", which is reflective upon itself:

<SemanticTypeDecl OfType="Noun">
  <!-- The ST has a Name attribute because it's Struct defines it. -->
  <AttributeValues>
    <Attribute Name="Name" Value="Noun"/>
  </AttributeValues>
</SemanticTypeDecl>

<SemanticTypeStruct DeclType="Noun">
  <Attributes>
    <!-- here the attribute Name's value is the implementing field name -->
    <!-- For example, in C#, resulting form is "string Name;" -->
    <NativeType Name="Name" ImplementingType="string"/>
  </Attributes>
</SemanticTypeStruct>

A semantic type has:

  • a declaration
  • a structure

In the case of the type "Noun", the attribute "Name" is reflective on the native type "Name" that is implemented by the language type "string."

Here is the declaration for the semantic type "DebugMessage" used in the heartbeat example:

<SemanticTypeDecl OfType="Noun">
  <AttributeValues>
    <Attribute Name="Name" Value="DebugMessage"/>
  </AttributeValues>
</SemanticTypeDecl>

<SemanticTypeStruct DeclType="DebugMessage">
  <Attributes>
    <NativeType Name="Message" ImplementingType="string"/>
  </Attributes>
</SemanticTypeStruct>

This can be read as: "We have a semantic type named 'DebugMessage' consisting of a type 'Message' implemented by the type 'string' ."

Ideally, this should have been implemented as a hierarchy, where Message would itself be a semantic type called "Text" implemented by "string" and DebugMessage would be implemented with the semantic type "Text" rather than the native type "string." The protocols that HOPE currently describes will eventually be refactored to be purer with the semantic type system that underlies that HOPE architecture.

Code Generation

The XML declaring the semantic types is converted into C# code (viewable in the HOPE application). For example, here is the code fragment for the semantic types described above:

// This code has been generated by the Semantic Type System.
using System;
using System.Drawing;
using System.Collections.Generic;
using Clifton.SemanticTypeSystem.Interfaces;
using Clifton.Receptor.Interfaces;

namespace SemanticTypes
{
  //Implements ISemanticType as the object type and requires Initialize().
  public class Noun : ISemanticType
  {
    //Native types.
    public string Name {get; set;}

    //Constructor. Typically called by reflection.
    public Noun() {}

    //Implements ICreate.
    public void Initialize(ISemanticTypeSystem sts)
    {
      //Native type initializers.
      Name = "Noun";
      //Additional semantic type initializers.
    }
  }

  //Implements ISemanticType as the object type and requires Initialize().
  public class DebugMessage : ISemanticType
  {
    //Native types.
    public string Message {get; set;}

    //Constructor. Typically called by reflection.
    public DebugMessage() {}

    //Implements ICreate.
    public void Initialize(ISemanticTypeSystem sts)
    {
      //Native type initializers.
      //Additional semantic type initializers.
    }
  }
...

As new types are recognized, the application generates the code and compiles them, placing them into a type repository. A desired type can then be instantiated through a request on the type system:

dynamic signal = SemanticTypeSystem.Create("SystemMessage");

Compile-Time Code Generation

It is certainly reasonable that the type can be fully known even at runtime by participating receptors, so for that reason, one could extend the system to implement interfaces in a separate assembly that define the compile-time characteristics of the type. This would remove the duck-typing but requires an agreement between producer and consumer receptors that foreknowledge of the type exists. Supporting this compile-time "during receptor coding" behavior will be added later.

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

About the Author

Marc Clifton

United States United States
Marc is the creator of two open source projets, MyXaml, a declarative (XML) instantiation engine and the Advanced Unit Testing framework, and Interacx, a commercial n-tier RAD application suite.  Visit his website, www.marcclifton.com, where you will find many of his articles and his blog.
 
Marc lives in Philmont, NY.

Comments and Discussions

 
QuestionAny chance of rewriting the introduction? PinpremiumQwertie19-Jul-14 12:23 
AnswerRe: Any chance of rewriting the introduction? PinprotectorMarc Clifton21-Jul-14 4:14 
GeneralMy vote of 5 PinpremiumPaw Jershauge5-Jun-14 12:26 
QuestionA CEP System? PinprofessionalCatchExAs4-Jun-14 2:55 
AnswerRe: A CEP System? PinprotectorMarc Clifton4-Jun-14 3:07 
GeneralRe: A CEP System? PinprofessionalCatchExAs4-Jun-14 5:00 
GeneralRe: A CEP System? PinprotectorMarc Clifton4-Jun-14 5:10 
QuestionPossible Uses PinmemberTim Carmichael27-May-14 2:33 
AnswerRe: Possible Uses PinprotectorMarc Clifton27-May-14 5:24 
GeneralRe: Possible Uses PinmemberTim Carmichael27-May-14 5:30 
GeneralRe: Possible Uses PinprotectorMarc Clifton27-May-14 9:58 
GeneralRe: Possible Uses PinmemberTim Carmichael27-May-14 11:48 
QuestionWhat in the world is this craziness?? (My vote of 5) PinmemberJaredThirsk26-May-14 19:54 
AnswerRe: What in the world is this craziness?? (My vote of 5) PinprotectorMarc Clifton27-May-14 5:25 
GeneralInsane Pinmember d@nish 26-May-14 19:08 
QuestionNice article! PinpremiumVolynsky Alex26-May-14 8:39 
AnswerRe: Nice article! PinprotectorMarc Clifton26-May-14 8:42 
GeneralRe: Nice article! PinpremiumVolynsky Alex26-May-14 8:45 
QuestionExcellent article PinpremiumMike Hankey26-May-14 5:37 
AnswerRe: Excellent article PinprotectorMarc Clifton26-May-14 8:42 
GeneralMy vote of 5 PinpremiumPaulo Zemek26-May-14 5:30 
GeneralRe: My vote of 5 PinprotectorMarc Clifton26-May-14 8:42 
QuestionBrilliant Marc - thanks PinmemberGarth J Lancaster26-May-14 5:21 
AnswerRe: Brilliant Marc - thanks PinprotectorMarc Clifton26-May-14 5:26 
QuestionSmall detail... PinpremiumPaulo Zemek26-May-14 5:19 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

| Advertise | Privacy | Mobile
Web02 | 2.8.140721.1 | Last Updated 10 Jul 2014
Article Copyright 2014 by Marc Clifton
Everything else Copyright © CodeProject, 1999-2014
Terms of Service
Layout: fixed | fluid