RAPPTOR.Persistence - Transparent object persistence the easy way

• Download source files - 453 Kb

Introduction

In application development - as every developer knows - there is always a point in time where he has to deal with object-persistence. So if you don't want to write SQL-code or stuff like that all the time, you need a general solution - an object persistence framework. Such frameworks have very different approach and are often specialized for different tasks and environments.

I'm a contributor to an Open source-project that implements a web-application-server called RAPPTOR[^]. There, we had to deal with that nasty persistence-thing. But when we took a look at the market, there was no solution that satisfied all our needs.

The needs in short

• Open source-license (LGPL to enable commercial projects based on our work)
• Automatic schema-evaluation (prevents from writing SQL-code or things like that)
• Runtime schema-alteration (we want runtime-loadable plugins empower to establish their own needed data structures)
• Database-independence (we want to be able to host our server in as many environments as possible)
• Mono-compatible

There are many powerful frameworks around, but we couldn't use any of them without any restrictions and so we decided to realize it ourselves. Because our web-application-server is called RAPPTOR, we decided to name our persistence framework RAPPTOR.Persistence. It's completely independent from the application-server, so everybody is invited to use it. In this article, I want to give an overview on how to use that thing.

How it works

The general philosophy of our framework is that the storage-schema follows our object-model and not vice versa. So, it would not be possible to set an object model on top of an existing SQL-schema. We first want to write our code and then the framework will do the data-storage stuff for us.

I will now explain in short the general structure of the framework. It is divided into three layers:

• the data-model layer
• the general database layer
• the relational database layer

The data-model layer is a very abstract thing that enables us to implement different drivers to deal with data from our code in different ways. Currently, there is only one implementation called DataModelREF that reflects an object-model to retrieve the schema and uses specific member declaration helper-classes to read and write the data of our object-model. That data-model (that's the way we call such drivers) is the one I will introduce in a later example. So it is not that schema information must come from a class hierarchy - e.g. an XML-schema with associated data could be persisted by a suitable model-driver, too.

The general database layer is used by the data-model drivers to access stored data in general. It is designed in a way that enables us to write a driver for an object-oriented database system. General object-management and caching are realized here.

The relational database layer sits on top of the general database layer and specializes it towards relational databases. The design-goal was to be able to write drivers for relational databases in a very easy way. Currently there are three drivers supporting MySQL 4.1, PostgreSQL and MSSQL.

The whole system is implemented by using the new .NET concepts from the 2.0 Framework. So you need either mono SVN[^] or the .NET 2.0 (beta) Framework.

Now, I will show you a small example. With the help of an easy object I will show you what happens in the database. After that I will change the structure a little bit. Finally, in the third part we will enhance complexity by expanding the data model using inheritance. By using such a persistence layer, life becomes much easier for developers.

The beginning

Before we can work on the persistence layer and build our object model we have to prepare the database. We need PostgreSQL in our example, the actual version is 8.0. In the PostgreSQL admin tool or the phpPgAdmin I'll generate an empty database and call it "CarData". Consider that you have all the needed privileges to that database, especially CREATE, DROP and ALTER rights on tables. Preparation on the database side is finished with that.

The easiest way to demonstrate the example is to take a console-project and add references to "RAPPTOR.Persistence.dll" and "RAPPTOR.Persistence.PostgreSQLDriver.dll".

Define the persistent class

We have to take a nice example. So we have to store something that everybody knows: cars. Objects from this class have certain properties, e.g. manufacturer, a color, max speed and the number of gears.

In our project we add a new class "cars.cs". In the using directive we add:

using Rapptor.Stage0.DataModel.REF;

In order to make the class Car persistent, we have to inherit from the class ManagedObject. Furthermore our class is complete with maintenance functionally and becomes an object ID (ObjID) - in RAPPTOR this is a global unique identifier (GUID).

If you want to store members in the database you define them by special data types. These generic classes are defined in the DataModel.REF namespace and are inherited from the MemberManager class. You can use the following data types:

EV<T>	Embedded value; simple value type
RO<T>	Referenced object; reference of a complex object
EO<T>	Embedded object; embedded complex object
COL<T>	Collection; list of embedded complex objects

As we use .NET generics we will work with their type secure members. We have to use the methods Value and Object to read and write data (we will see that later in an example). Complex objects are the ones that are inherited from the class ManagedObject.

We define the following members: the Manufacturer and Color as string, the MaxSpeed as float and the number of Gears as byte. In the code it looks like this:

public class Car : ManagedObject
{
  public EV<string> Manufacturer;
  public EV<string> Color;
  public EV<float> MaxSpeed;
  public EV<byte> Gears;
}

The main program

In our main program we make an instance of the class Car. We have to use the following namespaces:

using Rapptor.Stage0.DataModel;
using Rapptor.Stage0.DBDriver.PostgreSQL;

First, we define a handle to the database. In the static GlobalDataModel we join the driver with the class member DB. Then we make an instance of DBDriverPostgreSQL and call the Open method. The connection string is the parameter; use it like an ADO.NET string:

server	instance of PostgreSQL server
database	database name ("CarData")
uid	username, who has rights on the database
pwd	password of the user

Finally the PostgreSQL driver object has to be set on the static member DB of the class GlobalDataModel.

DBDriverPostgreSQL db;
db = new DBDriverPostgreSQL();
db.Open(
  "server=yourserver;database=CarData;uid=testuser;pwd=");
GlobalDataModel.DB = db;

Here comes our car. We create a green Porsche which is stored in the variable car:

Car car = new Car();
car.Manufacturer.Value = "Porsche";
car.Color.Value = "green";
car.MaxSpeed.Value = 250;
car.Gears.Value = 5;

You have to use the property Value, it is mandatory (handled in DataModelREF). The reward is a type secure programming. Now you can run the program and - nothing happens visibly. The local object car in the memory is forgotten when the program ends. Look at the next chapter and you will see some interesting things in the background.

Behind the scene - a look at the database

The tables in the database "CarData" are automatically generated by the framework. At the moment the PostgreSQL driver object is associated with the GlobalDataModel. The database structures are compared and if the database is empty, the system creates all the necessary tables. Every persisted class has one table (look at "dn_car"). The columns represent the members of the class. Every object is also stored in the table "object" which saves further information, e.g. the column "TypeID" saves the class membership. Finally the "globalvalues" table holds the entire data model structure as a serialized SOAP stream in order to compare the structures faster.

Take a look at the database:

All columns in the table "dn_car" represents the class Car. What I declared before the member "OID" is the system administrated unique object ID. So every object is identified clearly:

If the green Porsche is instantiated, the object is stored in a memory cache managed by the framework. Later the object is stored in the database, managed by the GarbageCollector. This happens when there is no reference to the object, like when the program terminates. In the table "dn_car" you will see:

Even the table "object" has an entry:

The next step: changing the structure

I said that cars have a model description. Now we can expand the class structure and insert a string member Model, as you can see in the next code block:

public class Car : ManagedObject
{
  public EV<string> Manufacturer;
  public EV<string> Model;
  public EV<string> Color;
  public EV<float> MaxSpeed;
  public EV<byte> Gears;
}

In the main program where we create the new car we will set the model to "911" and make it red.

Car car = new Car();
car.Manufacturer.Value = "Porsche";
car.Model.Value = "911";
car.Color.Value = "red";
car.MaxSpeed.Value = 260;
car.Gears.Value = 5;

We know that we have to look directly into the database after the program starts. The table "dn_car" has changed; a new column "model" has been inserted by the framework. This happens by comparison of the data structures:

What happens to our values in the table? The green Porsche still exists; the red one has been added. Only the red Porsche has a model description, formerly the member wasn’t known.

As you can see, changing the data structure lately has no problem. You don't have to care about that, the database structures are updated automatically. The algorithm will do that by the principle "keep the values of the members that still have the same name and data-type".

Look at the table "dn_car":

Inheritance and complex structures

Now we will enlarge the car model.

Referenced objects

In the model, we save the car driver. Every car refers to a driver. Add the class Driver as shown:

public class Driver : ManagedObject
{
  public EV<string> Name;
  public EV<DateTime> Birthday;
}

Now we add a reference to the Car class:

public class Car : ManagedObject
{
  ...
  public RO<Driver> CarDriver;
}

Inheritance

We also want to save racing cars with a notable engine in the model. We inherit the class RacingCar from Car because a racing car has all the properties of a car. As Car inherits from ManagedObject, RacingCar is also a persistable class.

Embedded objects

We want to embed an engine object in our racing car. So define the class Engine as follows and use the class RacingCar as declared by inheritance:

public class Engine : ManagedObject
{
  public EV<float> Cylinder;
  public EV<float> HorsePower;
}

public class RacingCar : Car
{
  public EO<Engine> CarEngine;
}

Collections

You know, racing cars are built using advertisement-money. All the sponsors of the car will be managed by a collection:

public class Sponsor : ManagedObject
{
  public EV<string> Name;
  public EV<double> AdvertisementMoney;
}

Now complete the class RacingCar and give it a chief designer:

public class RacingCar : Car
{
  public COL<Sponsor> Sponsors;
  public EO<Engine> CarEngine;
  public EV<string> ChiefDesigner;
}

The object model shows the structure, the code block demonstrates how to define the objects:

Driver cd = new Driver();
cd.Name.Value = "Michael Schumacher";
cd.Birthday.Value = System.DateTime.Parse("01/03/1969");

Engine engine = new Engine();
engine.Cylinder.Value = 10;
engine.HorsePower.Value = 900;

Sponsor s1 = new Sponsor();
s1.Name.Value = "Red Flash";
s1.AdvertisingMoney.Value = 1000000;

Sponsor s2 = new Sponsor();
s2.Name.Value = "MoneyBank";
s2.AdvertisingMoney.Value = 5000000;

RacingCar rc = new RacingCar();
rc.Manufacturer.Value = "Ferrari";
rc.Model.Value = "F2005";
rc.Color.Value = "red";
rc.MaxSpeed.Value = 380;
rc.Gears.Value = 6;
rc.CarDriver.Object = cd;
rc.ChiefDesigner.Value = "Rory Byrne";
rc.CarEngine.Object = engine;
rc.Sponsors.Add(s1);
rc.Sponsors.Add(s2);

The database-schema that establishes while the program runs, represents our object-model. Every embedded and referenced object is persisted with its actual values.

The RacingCar-object rc is now stored in the tables "dn_car" and "dn_racingcar" (look at the OID). To store objects using inheritance, every inheritance-layer of an object (a class) is stored in a table. To collect all the members of a RacingCar object, the tables "dn_car" and "dn_racingcar" are connected 1:1 by the OID.

Every object is represented in the "object"-table. Here the two sponsors and the engine are very interesting, because all of them are referencing to their parent object by the column "ParentID". The column "MemberInParent" additionally holds the name of the embedding member of the parent object. The multiple entry "Sponsors" indicates a collection as the holding member.

What's next?

In the next tutorial I will explain, how to load stored objects from the database. This tutorial only shows you the general function and the central idea of the framework.

At the moment RAPPTOR.Persistence is in its development phase and uses - as it is very young - very new approaches from the .NET 2.0 Framework - especially generics. To use it you need to have .NET 2.0 (at the moment a beta 2) or mono 1.1.8. The persistence layer is in alpha-phase and is growing fast. For example there will be many more service-functions in the DataModelREF-classes, a query language and so on. We cannot guarantee that it will work without any problems. It has a huge optimization potential. If you are interested in helping us to extend, polish or test the framework please contact us[^] or Novell Forge[^]. Every contribution is welcome.