Object-oriented database programming with db4o - Part 2

• Download oodb_db4o_part_2_demo - 595.25 KB

Contents

• Introduction
• Client-server Deployment
  • Embedded Server Mode
  • Networking Mode
  • Out-of-Band Signaling
• Transaction & Concurrency
  • Transaction Support
  • Semaphores
• Conclusion
• Resources

Introduction

In part 1 of this article I discussed how to perform CRUD operations with db4o. (If you have not read through part 1, I suggest you to do so to learn the basics of db4o as well as understand the object model used in the code samples.)

In this part, I will explore other features of db4o that are useful in multi-threading environment and n-tier applications. These include client-server feature and transaction & concurrency support. Before we start, notice that the development version of db4o 7.2 (which can be downloaded here) and C# 3.0 are used for all code samples.

Client-Server Deployment

In part 1, we’ve seen the simplest usages of db4o database, i.e. the stand-alone mode in which we create an instance of type IObjectContainer via the Db4oFactory and interact with that instance. While that usage mode is sufficient for single-threaded standalone or embedded applications, it will not work for applications in which multiple clients concurrently interact with the same database or applications in which the database client and data server reside in different machines. In fact, the database file is exclusively locked by the IObjectContainer object created in the standalone mode so that no other instance can access to it, less make modification to it. Fortunately, db4o does offer solution for this via its client-server feature which includes two setup modes, embedded server and networking.

In the embedded server mode, a single instance of type IObjectServer is used to create multiple IObjectContainer instances. All of these objects reside in the same process and there’s no TCP/IP involved in the communication (that’s why ‘embedded’). Let’s look at some code demonstrating this setting.

IObjectServer server = null;
try
{
    // Create the server instance
    server = Db4oFactory.OpenServer(DB_PATH, 0);

    // Create the first client via the server instance
    using (IObjectContainer client1 = server.OpenClient())
    {
        Line line = client1.Query(l => { return l.Color == Color.Green; })[0];
        line.Color = Color.Black;

        // Store... then try to access the line in another client
        client1.Store(line);
        using (IObjectContainer client2 = server.OpenClient())
        {
            // Nothing is found because the transaction is not committed
            Assert.AreEqual(
                0,
                client2.Query(l => { return l.Color == Color.Black; }).Count
            );
        }

        // Now, commit
        client1.Commit();
        using (IObjectContainer client2 = server.OpenClient())
        {
            // The new line is found now
            Assert.AreEqual(
                1,
                client2.Query(l => { return l.Color == Color.Black; }).Count
            );
        }
    }
}
finally
{
    if (server != null)
        server.Close();
}
   

The above code shows that we can have multiple clients accessing the same database file. The second parameter of Db4oFactory.OpenServer() method is the port number and we pass 0 to tell db4o that the embedded server mode is used instead of the networking mode. The code also shows how the changes made by one client impact the other client. Specifically, before client1 commits its changes, client2 will not see any update at all. (We will discuss this further in the Transaction section.)

To make the interaction between the two clients easy to understand, I just use one thread to launch the two clients. In real-life applications, it’s likely that you will create multiple threads each of which is responsible for one or more database clients. (In fact, if there’s no such use case then you should use the standalone mode of db4o instead of the embedded mode.)

If your database is deployed in a different process or machine from the database clients, you would need to use db4o’s networking mode instead. There are a few differences when using this mode with the embedded server mode:

• You will use Db4oFactory.OpenClient() instead of IObjectServer#OpenClient(). This makes sense because your client code does not have access to any IObjectServer which is created on the server side.
• You need to specify a valid port on which the server listens and pass it as a parameter of Db4oFactory.OpenClient().
• Finally, the server must explicitly grant access to some credentials and the client must use one of these credentials to access to the server.

Simple enough, let’s look at some code

IObjectServer server = null;
try
{
    // Create a server socket and have it listen on port 9999
    server = Db4oFactory.OpenServer(DB_PATH, 9999);

    // Specify some permitted credentials
    server.GrantAccess("buuid", "buupass");
    server.GrantAccess("somebody", "somepassword");

    try
    {
        // Try to connect using the invalid credential
        IObjectContainer client = Db4oFactory.OpenClient("localhost", 9999, "buuid", "any");
        Assert.Fail("Wrong password, must not go here");
    }
    catch (Exception e) 
    {
        Assert.IsTrue(e is InvalidPasswordException);
    }

    using (IObjectContainer client1 = Db4oFactory.OpenClient("localhost", 9999, "buuid", "buupass"))
    using (IObjectContainer client2 = Db4oFactory.OpenClient("localhost", 9999, "somebody", "somepassword"))
    {
        client1.Store(new Line(new CPoint(0, 0), new CPoint(10, 10), Color.YellowGreen));
        client1.Commit();

        // client2 will see the change
        Assert.AreEqual(
               1,
               (    from Line line in client2
                    where line.Color == Color.YellowGreen
                    select line
               ).Count()           
        );
    }
}
finally
{
    if (server != null)
        server.Close();
}

Again, the code is much simplified by having the server and its two clients run on the same thread. If you move the server to another process or machine then the behavior will remain unchanged. The only other point of interest in the code is that the query is written in LINQ just to show off db4o’s LINQ provider which is available in db4o 7.2.

In the networking mode, sometimes you will want the client to be able to request the server to perform an arbitrary data-related action not existing in any IObjectContainer’s API. Closing the server is one example: while you can call IObjectServer#Close() to shut-down your database server, there’s no way to do that via the IObjectContainer interface, which is the only thing accessible by the client code. While you can do this by creating a service interface in front of the database component, the more time-effective alternative is to leverage db4o’s out-of-band signaling feature.

In order to have the server respond to a custom message from the client, you simply need to write a class implementing the IMessageRecipient interface which has a single method, ProcessMessage(IMessageContext context, object message). (The message parameter is the actually message object that your client code will send to the server.) You will then register an instance of this interface by passing it as the parameter into the IObjectServer#SetMessageRecipient() method. (IMO, db4o should have an overload of this method receiving a delegate instead so that developers don’t have to write an interface implementation.) On the client side, you will use an instance of IMessageSender retrieved from the IObjectContainer in order to send a message to the server.

Below is a sample code which does just that.

[TestClass]
public class OutOfBandTest : IMessageRecipient
{
    public const string DB_PATH = "mypaintdb.db";
    IObjectServer server = null;

    [TestMethod]
    public void TestCloseServer()
    {
        // Start the server and set initialize message recipient
        server = Db4oFactory.OpenServer(DB_PATH, 9999);
        server.GrantAccess("buuid", "buupass");
        server.Ext().Configure().ClientServer().SetMessageRecipient(this);

        using (IObjectContainer client = Db4oFactory.OpenClient("localhost", 9999, "buuid", "buupass"))
        {
            IMessageSender sender = client.Ext().Configure().ClientServer().GetMessageSender();

            // Send the shutdown message to server
            sender.Send(new ShutdownServerMessage());
            try
            {
                client.Store(new Line(null, null));
                Assert.Fail("Server is already shutdown.  Must not go here.");
            }
            catch (Exception e)
            {
                Assert.IsTrue(e is DatabaseClosedException);
            }
        }
    }

    public void ProcessMessage(IMessageContext context, object message)
    {
        if (server != null && message is ShutdownServerMessage)
        {
            server.Close();
        }
    }

    class ShutdownServerMessage { }
}

That’s it for working with client-server features of db4o. Let’s now discuss about db4o’s support for transaction.

Transaction and Concurrency

So far, in most of our examples, we have used db4o’s implicit transaction which automatically starts a new transaction whenever an object container is created via Db4oFactory.OpenFile() or IObjectServer#OpenClient() and commits the changes when the container is closed (or disposed). Instead of having db4o automatically control the transaction, we can use the Commit() and RollBack() methods of IObjectContainer to manually manage db4o transactions.

Now, suppose we need to write a function to change the color all of existing line shapes in our database and we need to assure that this operation is done atomically, i.e. either all lines are updated or no line is; explicit transaction management would be absolutely necessary here. Let's look at the code that does just that.

IObjectContainer container = null;
try
{
    container = Db4oFactory.OpenFile(DB_PATH);
    IObjectSet set = container.QueryByExample(typeof(Line));
    Assert.IsTrue(set.Count > 0);

    // Change color of all lines
    for (var i = 0; i < set.Count; i++)
    {
        ((Line)set[i]).Color = Color.Purple;
        container.Store(set[i]);

        // After storing 2 lines, deliberately fail
        if (i == 1)
        {
            throw new Exception();
        }
    }
    container.Commit();
}
catch
{
    container.Rollback();

    // Strange enough, this asserts to true...
    Assert.AreEqual(
        2, 
        container.Query(line => {return line.Color == Color.Purple;}).Count
    );
}
finally
{
    container.Close();
}

using (container = Db4oFactory.OpenFile(DB_PATH))
{
    Assert.AreEqual(
        0,
        container.Query(line => { return line.Color == Color.Purple; }).Count
    );
}

You can see that I deliberately throw an exception after two lines have their colors changed. Interestingly, the assertion code after the rollback indicates that there are actually two lines having their color changed. Isn’t this unexpected because the rollback is supposed to wipe out all changes in the transaction? Well, remember in part 1 of this article, I mentioned about the db4o’s object cache which maintains references to all objects stored or retrieved in the life-cycle of an object container. Back to our example, when the container fetches the list of line shapes from the database, it notices that some lines already existing in its cache and returns those cached references instead of creating and returning new instances. In other words, while the rollback operation is successful and the database is updated properly with the wipe-out (otherwise, we would receive an exception), the returned objects are those having been in memory instead of new instances created from data retrieved from the database. To retrieve the database copies, we can simply perform the query in a new container as you can see in the code sample. An alternative is to clear the cache by invoking IObjectContainer#Ext().Purge() before running the query.

So that we have seen how to explicitly use Commit() and Rollback() to assure the atomicity of series of data operations, let’s look at how db4o assures the data integrity in case of multiple concurrent accesses to the same database in the client-server mode. Recall the code sample of the Embedded Server part, client2 can only see the changes made by client1 after client1 invokes the IObjectContainer#Commit() method. And that’s exactly how db4o controls concurrency: the Read Committed isolation level is used for all db4o’s transactions. In this isolation level, a client cannot see uncommitted changes made by another client and therefore there’s no way a client can operate on data which are going to be rolled back (e.g. a “dirty read”). On the other hand, this isolation level is not strong enough to prevent “non-repeatable read” and “phantom read” and we have to explicit write code in the application to handle these scenarios since db4o currently only supports Read Committed isolation level. Because the built-in Read Committed isolation level of db4o is already demonstrated in the code sample for Embedded Server, I won’t provide another here. Instead, we will look at how to use db4o’s semaphores to control transaction isolation in application code.

Db4o provides semaphores to allow developers to control access to critical sections of the application code. A semaphore is identified by a name and can be acquired by an object container by invoking IObjectContainer#Ext().SetSemaphore(string name, int waitForAvailability), which returns true if the semaphore is successfully acquired and false otherwise. The waitForAvailability parameter indicates the time in millisecond that the method needs to return in case the semaphore is already acquired by another container and 0 means an immediate return. When an object container successfully acquires a semaphore then no other containers can acquire it until the semaphore is released by invoking IObjectContainer#Ext().ReleaseSemaphore(string name). The use of semaphore is a kind of pessimistic locking strategy and you can imitate even Serializable isolation level by explicitly acquiring the semaphore before performing any data operation via an object container.

Now, let’s write a client-server code which has two clients, one retrieves all Line shapes and try to change their color and another client running on another thread tries to update the color of a specific line. Let’s assume there is a requirement that client2 can’t perform any update while client1 is on its way updating all lines. (Bear with me; it’s hard to think of good concurrent scenarios for this simple paint domain and I’m too lazy to create another object model just to demonstrate the semaphore feature.)

IObjectServer server = null;
try
{
    server = Db4oFactory.OpenServer(DB_PATH, 9999);
    server.GrantAccess("buuid", "buupass");

    // This client updates the color of all lines
    Thread clientThread1 = new Thread(delegate()
        {
            using (IObjectContainer client1 = Db4oFactory.OpenClient("localhost", 9999, "buuid", "buupass"))
            {
                // Acquire the semaphore
                if (client1.Ext().SetSemaphore(LOCK_KEY, 0))
                {
                    IObjectSet set = client1.QueryByExample(typeof(Line));
                    foreach (Line line in set)
                    {
                        line.Color = Color.RosyBrown;
                        client1.Store(line);
                        Thread.Sleep(500);
                    }
                }
            }
        });

    // This client tries to delete the first line
    Thread clientThread2 = new Thread(delegate()
    {
        using (IObjectContainer client2 = Db4oFactory.OpenClient("localhost", 9999, "buuid", "buupass"))
        {
            // Cannot acquire semaphore with this call
            Assert.IsFalse(client2.Ext().SetSemaphore(LOCK_KEY, 0));

            // After 5 seconds, client1 already releases the lock (there're only 4 Lines in the DB)
            // thus, this call would succeed
            Assert.IsTrue(client2.Ext().SetSemaphore(LOCK_KEY, 5000));

            // Test another call - pass because client2 is already the lock owner
            if (client2.Ext().SetSemaphore(LOCK_KEY, 0))
            {
                IObjectSet set = client2.QueryByExample(typeof(Line));
                ((Line)set[0]).Color = Color.Black;
                client2.Store(set[0]);
            }
        }
    });

    // Start client 1, then sleep a bit to make sure the semaphore is acquired
    // before starting client 2
    clientThread1.Start();
    Thread.Sleep(1000);
    clientThread2.Start();

    // Wait for the threads to terminate...
    clientThread1.Join();
    clientThread2.Join();
}
finally
{
    if (server != null)
        server.Close();
}

Acute readers will ask, “Hey, client1 never releases the semaphore since it does not invoke IObjectContainer#Ext().ReleaseSemaphore(). How come client2 can acquire the semaphore?” Good question. I intentionally did that to demonstrate another interesting point about db4o’s semaphore: when the client object container is closed, the server will detect the connection drop and free up all semaphores previously acquired by that client. However, that’s for demonstration purpose only and I suggest developers to explicitly release the semaphore in application code instead of relying on the db4o’s connection detection mechanism which can be time-undeterministic in case of a client crash.

Conclusion

We have discussed through the client-server feature and transaction & concurrency support in db4o. This is not an end though, since db4o still has so many interesting features that are worth looking at. However the two features introduced in this article, together with the knowledge presented in part 1, should provide you with enough information to use db4o in more advanced scenarios than just in standalone or embedded applications. I hope you enjoy reading the article!

Resources

• Object-oriented database programming with db4o - Part 1
• Db4o 7.2 development version download page