Legion: Build your own virtual super computer with Silverlight

• Download Legion for VS2008 Silverlight 2 - 966 KB

Contents

• Introduction
• Background
• Long live Clog
• Legion Agent UI
• Legion Manager
• Components Overview
• Grid Tasks
• Configuration
• Agent Performance Metrics
• Prime Finder Task
• Agent Task Execution
• Client-side Mutual Exclusion
• Expiring Collections
• Saving and retrieving data using the Isolated Store
  • Saving data using the Isolated Store
  • Retrieving data using the Isolated Store
• How to write a Grid Task
• Future enhancements
• Conclusion
• Special Thanks
• References
• History

Introduction

And he asked him, What is thy name? And he answered, saying, My name is Legion: for we are many.

Mark 5:9 (KJV)

I have always been a fan of distributed computing. With the advent of Silverlight, I felt there was an exciting opportunity to create a Grid computing framework utilizing the client-side Silverlight CLR. That is how Legion emerged.

Legion is a Grid Computing framework that uses the Silverlight CLR to execute user definable tasks. Legion uses an ASP.NET application and web services to download tasks, upload result data, and provide grid-wide thread-safe operations for web clients or agents. Multiple tasks can be hosted at once, with Legion managing the delegation of tasks to agents. Client performance metrics, such as bandwidth and processor speed, may be used to tailor jobs for clients. Legion provides a management service and WPF application that is used to monitor the Legion grid.

I have deployed Legion to a demonstration server here so you can see it in action.

Figure: Legion system overview.

Background

While Grid computing is not new, there has been a resurgence of interest in it due mainly to organisations looking to capitalise on their underutilised IT infrastructure, to perform computationally intensive business related tasks.

Aside from large organisations jumping on the bandwagon, public interest in Grid computing has of course been stimulated by large scientific projects, such as the Seti@home project. After performing some initial research for this article I became enthused by what a volunteer grid computer is capable of achieving, and what, by offering a framework like Legion may achieve. The results of some volunteer Grids have been remarkable. For example, the Smallpox Research Grid Project managed to find 44 strong treatment candidates for smallpox; an, as yet, incurable disease. Vaccination for Smallpox ended in the 1980s, but there are fears that the virus may re-emerge. (Wikipedia, 2007; The Inquirer, 2003)

The allure of harnessing unused computing power is strong, and so too is the notion of many users contributing otherwise wasted clock cycles for the benefit of humanity. One reason why so many Grid projects have garnered so many participants, is the feeling of belonging to a shared mission; it's free, and users gain a feeling of contributing to a good cause, just by running an application in the background on their computer.

Could it be that we are on the cusp of a web paradigm shift, where web browsers share the load of the web server? Opening the door to smaller start-ups with little investment in infrastructure, providing an unlimited capacity; serving any number of clients. Such a capability has traditionally been the domain of monolithic companies with massive data centres. What would that do? Well, it would radically transform the web; providing a level of democratisation never before seen. Taking this idea further, imagine, if you will, what a peer to peer WebTorrent protocol might bring. Web servers and web browsers alike, serving as seeds and distributing the load. We are venturing outside the scope of the article a little, but I raise these ideas to encourage in the reader a feeling of opportunity. For that is what having a CLR in the browser now affords us. An opportunity, not only to create nice UI, but also to move to a less server-oriented application model. Not merely a thin-client model, but rather a distributed processing model.

I believe security to be the main challenge in providing client-centric peer to peer volunteer computing. Protecting a user's privacy is paramount, and it's risky sending sensitive information, which may be eavesdropped or modified by some rogue client. Likewise, trusting the results from an inherently untrustworthy source presents an issue. Clearly mechanisms are required to address these issues.

Likewise, providing secure cross domain browser support will bring a lot of opportunity to decentralise and share services more readily between organisations. Microsoft and others have proposed some techniques. Yet, it was inevitable that doing so within existing browser capabilities requires an assortment of hacks. Hacks, such as using IFrames to communicate across domains. While preventing cross domain browser communication seems reasonable, with today's ubiquity of webservices, we can quite easily send client data from server to server, and thus across domains, which effectively circumvents the protection mechanism anyway.

Unlike other distributed computing projects, Legion allows users to participate by simply viewing a webpage. The shift to cloud computing, and web based application, may have fostered a growing reluctance in users to download and install applications. Forcing users to download software to participate, while improving the dependability of the grid (we'll discuss the pitfalls of a browser based Grid later), must surely decrease the overall participation rate. Users prefer not to install software. It's a hassle.

Legion, on the other hand allows us to bring new 'volunteers' to our grid by means of providing enticing content. Thus, we are no longer dependent on a single motivating cause to bring participation. It provides developers with a system that will deploy and execute modules written using any .NET Desktop/Silverlight CLR compatible languages. The beauty of Legion is that it does not require user installation of any Legion software. It relies solely on the Silverlight CLR in the user's browser.

I've heard many comparisons between Silverlight, Flash, Java applets etc. Most comparison seems to resolve around what an old project manager of mine called "pointy clicky things."; pure UI. Well, in this article my aim is to take the focus away from being solely UI, and take more of a look at what having a .NET CLR in the browser might mean.

Long live Clog

Clog, unbeknownst to me, was a prelude to this project, and turned out to be invaluable in its development. Without Clog for Silverlight working on the concurrency elements of this project would have been made extremely difficult. I have, as it were, been eating my own dog food. It made debugging with multiple clients a breeze. Using the Log4NetLogStrategy I had logging coming in from the website, the Agents and even the WPF Manager at the same time. It would have been next to impossible to debug multiple clients executing concurrently. And using trace statement would have been awful. Clog is pretty cool, even if I do say so myself. I've made several important updates to its code base, which I will be publishing soon.

Legion Agent UI

On the client-side, Legion is hosted within a Silverlight module that automatically controls the downloading and execution of tasks.

Figure: Legion Agent pictured in browser.

The Agent visualizer that is provided may be customized to suit. I may provide a more discreet interface at a later stage. As can be seen, the Silverlight module is hosted on an HTML page, and will happily share this space with other content; be it HTML or perhaps some other interactive Silverlight animation to keep the user enthralled and the browser open.

The interface displays the task name, as it is defined in the server-side config, the number of tasks completed, stored on the clients machine in Isolated Storage, and the percentage complete of the current executing task.

Legion Manager

The Legion Manager is a WPF application that allows monitoring of the grid. It provides a summary of the progress of all tasks, and the current grid capacity; including available processing power in megaflops and total bandwidth for all connected clients.

Figure: Legion Grid Manager pictured in browser.

Components Overview

Legion consists of three main components. The GridComputing and GridComputing.Management components use the Desktop CLR. The third, GridComputing.Silverlight, is Legion's main client-side component, and is executed within the Silverlight CLR. The main types from each component are shown below.

Figure: Legion component overview.

The WPF Manager is a WPF application that communicates with the server-side GridComputing module to monitor the Legion grid. It consumes GridSummary instances that are created by the GridManager.

Agent and Client instances are passed to the grid web services as part of each request from the management and agent components; and they encapsulate the location data and performance metrics of callers.

The MasterTask and SlaveTask provide the respective server and client task implementations. Jobs are created by a MasterTask and dished out to SlaveTasks by the GridManager.

The GridMonitor is a tool to provide mutual exclusivity for Agents. The GridSync encapsulates the location of the code region, and the GridMonitor uses a web services to request exclusive access the region.

Agents communicate with Legion on the server-side using the JsonGridService. When working with Silverlight 1.1 we use JSON JavaScript Object Notation. You can find a little more info on working with Silverlight and web services in one of my previous articles.

Figure: JsonGridService class diagram.

The Grid service is mostly a wrapper for the GridManager with some additional error handling thrown in. The following is a brief overview of the JsonGridService methods:

• Register: Lets the server know that the calling Agent is ready to be connected to the network, and that it is available for new tasks. Returns a unique identifier that will be used in future calls to identify the Agent.
• StartNewTask: Retrieves a TaskDescriptor containing information regarding a new SlaveTask and the Job data required to run the task. This is called periodically by the Agent on the completion of each task.
• UpdateJobProgress: Lets the server know where the client is up to in its processing of a SlaveTask.
• JoinTask: Called when the SlaveTask completes on the client-side. This allows the result of the task to be processed by the MasterTask and joined with other results from other clients. This is the endpoint for a client's SlaveTask.
• Disconnect: Lets the server know that the Agent will no longer be processing tasks.
• Download100KB: Used when measuring client bandwidth. It simply returns a large object that brings up the SOAP message size to approximately 100 KB.
• LockEnter, LockUpdate, LockExit: Used to provide mutual exclusion for client-side code blocks. When an Agent has exclusive access, it periodically polls using the LockUpdate method to inform the LockManager that it is still active (that the client browser has not been closed).

Grid Tasks

When creating a new Legion task, we create a server-side master task, and a client-side slave task. A custom task is created by extending the MasterTask class in the main GridComputing module, and by extending the SlaveTask class in the GridComputing.Silverlight module. The role of the MasterTask is to split the grid computing activity into bite size chunks, and hand these off to client-side Agents. The client-side SlaveTask is the point of concurrency; this is where we place the code that we want executed by multiple clients. When a SlaveTask completes its work, the result is sent back to the MasterTask to be merged with the results of other SlaveTasks.

Figure: MasterTask class diagram.

The client-side static TaskRunner is responsible for the instantiation, asynchronous execution, and provision of Job data for the SlaveTask. The TaskRunner is also the main point of interaction between the Silverlight UI and Legion.

Figure: Silverlight Agent module - class diagram of TaskRunner and SlaveTask.

In order to execute a client-side task (a SlaveTask) we require the SlaveTask Type name, and the Job data. The Job data is created by the MasterTask, while the TypeName is defined in the server-side config as slaveType. The TaskDescriptor and child Job instance are returned from the JsonGridService when the Agent makes a call to JsonGridService.StartNewJob(Agent agent).

Figure: TaskDescriptor and Job class diagram.

Configuration

Legion uses a configSection to define the tasks to be run. First we declare the section within the configSections element, as shown in the following excerpt.

<configSections>
    <section name="Grid" 
        type="Orpius.GridComputing.Configuration.GridConfigSection" 
        requirePermission="false"/>
        
    <!-- Client Logging section -->
    <section name="ClientLogging" 
        type="Orpius.Logging.Configuration.ClientLoggingConfigSection"/>
    .
    .
    .
</configSections>

Then we declare our grid tasks within the section, as shown in the following excerpt.

<Grid>
    <Tasks>
        <!-- 
        slaveType: The name of the Silverlight task type.
            Please note that Version, Culture, 
            and PublicKeyToken are required.
        -->
        <add name="PrimeFinder"
             type="Orpius.GridComputing.Tasks.PrimeFinderMaster, 
                Orpius.GridComputing.Tasks.PrimeFinder"
             slaveType="Orpius.GridComputing.Tasks.PrimeFinderSlave, 
                Orpius.GridComputing.Tasks.PrimeFinder.Silverlight, 
                Version=1.0.0.0, Culture=neutral, PublicKeyToken=null"/>
        
        <add name="MutexExample" 
             type="Orpius.GridComputing.Tasks.MutexExampleMaster, 
                Orpius.GridComputing.Tasks.MutexExample"
             slaveType="Orpius.GridComputing.Tasks.MutexExampleSlave, 
                Orpius.GridComputing.Tasks.MutexExample.Silverlight, 
                Version=1.0.0.0, Culture=neutral, PublicKeyToken=null"/>
        
    </Tasks>
</Grid>

It is important to note that the slaveType attribute of each task must include the entire signature of the assembly. If we fail to include any part, such as the Version or Culture attributes, Silverlight will be unable to resolve the download path for the assembly.

Agent Performance Metrics

To measure the connection bandwidth we merely call a web service that returns a result message that is approximately 100 kilobytes. We do this once when the connection is established.

/// <summary>
/// Measures the bandwidth of the connection
/// with the server by calling a web service
/// method that returns a large message
/// of a predetermined size.
/// </summary>
/// <returns>The download rate of the client in KiloBytes.
/// </returns>
public static long MeasureBandwidthKBps()
{
    long resultDefault = 0;
    long result = resultDefault;

    JsonGridService gridService = JsonGridService.GetService();

    DateTime begun = DateTime.Now;

    try
    {
        byte[] download = gridService.Download100KB();
    }
    catch (Exception ex)
    {
        log.Debug("Unable to communicate with server.", ex);
        return resultDefault;
    }

    DateTime ended = DateTime.Now;

    TimeSpan span = ended - begun;
    if (span.TotalMilliseconds > 0)
    {
        result = 100000 / (long)span.TotalMilliseconds;
    }

    return result;
}

I used Fiddler to throttle the download rate by setting it to Simulate modem speeds in order to test the bandwidth measuring.

Figure: Setting Fiddler to simulate modem speeds.

We can see in the next capture that, following the throttling of the download rate, the result is 6 KB/s, which is about right. The accuracy is sufficient for our purposes. Of course we are only running the test once for each client, and so we must take the result as being just a rough estimate.

Figure: Debugging Agent with modem speed simulation.

To measure the client's processor speed, we have the client execute some floating point operations.

/// <summary>
/// Measures the processor speed in megaFLOPS.
/// Not particularly accurate, but it provides 
/// a rough indication of what the client can do.
/// </summary>
/// <returns>The processor speed in megaFLOPS.</returns>
public static long MeasureMFlops()
{
    double x = 0, y = 0, z = 0;
    DateTime startTime = DateTime.Now;
    /* 60 million fp operations? */
    for (int index = 0; index < 10000000; index++) 
    {
        x = 2.34 + index;
        y = 5.67 + index;
        z = (x * x) + (y * y) + index;
    }
    DateTime endTime = DateTime.Now;

    Console.WriteLine(x + y + z);

    TimeSpan span = endTime - startTime;

    return span.Ticks / 60;
}

When we prepare a job for an Agent in the server-side MasterTask we are then able to utilise the performance metrics in order to tailor a job best befitting the client. For example, if the client is on a slow connection or is unable to process a lot of operations, we can reduce the job size that we send to it.

Figure: Job tailored for Agent using performance metrics.

We can also make use of various other metrics, and client information. This is passed in the form of an IAgent instance to the MasterTask.GetRunData method.

Figure: IClient and IAgent interfaces. A MasterTask is able to use the client information to tailor a job for the Agent.

Prime Finder Task

Legion includes two demo tasks: Prime Finder, and Mutex Example. The first, Prime Finder, searches for prime numbers within a specified range. Each Agent is designated a range in which to test each number within the range for primality. When complete, the client-side SlaveTask returns the list of primes that were located, and the MasterTask inserts the results into a composite list. It's pretty simple.

/// <summary>
/// The client-side implementation of the Prime Finder task.
/// This task searches specified ranges for prime numbers
/// and then returns the results to the master.
/// </summary>
public class PrimeFinderSlave : SlaveTask
{
    static readonly ILog log 
        = LogManager.GetLog(typeof(PrimeFinderSlave));
    List<long> primes;

    public PrimeFinderSlave()
    {
        Run += AddNumberTaskClient_Run;
    }

    /// <summary>
    /// Handles the Run event of the AddNumberTaskClient control.
    /// This is where we do the processing for the task.
    /// </summary>
    void AddNumberTaskClient_Run(object sender, EventArgs e)
    {
        primes = new List<long>();

        long start = Descriptor.Job.Start;
        long end = Descriptor.Job.End;
        
        string infoMessage = string.Format(
            "PrimeFinderSlave starting. 
                Job Id is {0}. Range is {1} - {2}",
                Descriptor.Job.Id, start, end);
        log.Info(infoMessage);

        StepsCompleted = 0;
        StepsGoal = end - start;

        for (long i = start; i < end; i++)
        {
            if (IsPrime(i))
            {
                primes.Add(i);
            }
            StepsCompleted++;
            /* Sleep for a bit. */
            if (i % 5000 == 0)
            {
                int sleepTime = 200;
                System.Threading.Thread.Sleep(sleepTime);
            }
        }
        Result = primes;
    }

    static bool IsPrime(long candidate)
    {
        for (int d = 2; d <= candidate / 2; d++)
        {
            if (candidate % d == 0)
            {
                return false;
            }
        }
        return candidate > 1;
    }
}

Once the MasterTask is finished handing out search ranges to Agents, it continues to dispatch unjoined Jobs, until results are obtained for all.

Agent Task Execution

The SlaveTask is instantiated on the client-side using information present in the TaskDescriptor instance passed over the wire from the server-side GridManager. The TaskDescriptor contains a Job that is executed by the SlaveTask. The client-side TaskRunner is responsible for seeing that the task is downloaded and instanciated. And, it is the TaskRunner that oversees the running of the SlaveTask.

Figure: Task runner fetching a job.

The process of executing a task is outlined in the next diagram in more detail. Here we see that when the Silverlight Agent begins execution in a browser, it automatically initiates a client side task. The Agent registers with the Legion server, downloads the TaskDescriptor and begins work on the job.

Figure: Task execution.

When the client-side SlaveTask is complete, the result is merged by the MasterTask, and the TaskRunner requests a new task. Following the completion of all MasterTasks a Job with its Enabled property is passed back to the Agent indicating that there is nothing left to do. At this point the Agent will be in an idle state. Yet it will continue to periodically request tasks from the server in case new tasks become available.

Client-side Mutual Exclusion

In order to allow SlaveTask to work safely with shared data, we require a mechanism for mutual exclusivity. Just as .NET provides various types including the Monitor class for thread concurrency, I have created the GridMonitor class. With the GridMonitor we are able to safely access code regions asynchronously. The GridMonitor class uses the JsonGridService to control access to regions that are identified using GridSync instances. Each GridSync identifies one or more code regions that must only allow one client access at a time.

Figure: Silverlight Agent module, GridMonitor class diagram.

When an Agent wishes to gain exclusive access to a code block, it must make a request to the server-side LockManager using a GridMonitor in conjunction with a GridSync. The purpose of the GridSync is to identify the region, or regions, of the code that is/are to be synchronized. The ScopeTypeName property of the GridSync forms part of the identify of the GridSync, and will normally be the class type in which the GridSync is instanciated. The LocalName forms the second part of the identity, and is a name relative to the ScopeTypeName. The ScopeTypeName LocalName combination should generally be unique, otherwise deadlocks may ensue. It's analogous to using the lock (objectInstance) {...} syntax, and behaves in the same manner. Once a Client has ownership of a lock, it may execute the code block from whatever thread it likes. Thus, multithreading on the client still requires caution when using a GridMonitor and should employ the standard mutual exclusion techniques using Monitors etc. from the FCL.

Figure: Agent denied ownership of lock. GridMonitor will block until ownership is granted, or timeout occurs.

When using the GridMonitor, we wrap it in a using statement. When the executing thread leaves the using block, the GridMonitor will be disposed, and will attempt to release the lock by calling its own Exit method. This produces an asychronous call to the server-side LockManager; automatically relinquishing ownership of the lock.

While the GridMonitor has ownership of the lock, it will poll the server-side LockManager periodically, to let it know that the Agent is still alive, and that ownership of the lock should still remain with that Agent.

Figure: LockManager class diagram.

The following excerpt from the MutexExampleSlave task demonstrates how to create a mutually exclusive block using a GridMonitor.

/// <summary>
/// This is the client side implementation for a task
/// that demonstrates the use of the <see cref="GridMonitor"/>.
/// </summary>
public class MutexExampleSlave : SlaveTask
{
    static readonly ILog log 
        = LogManager.GetLog(typeof(MutexExampleSlave));
    const int iterationsCount = 10;

    GridSync exampleSync;
    
    public MutexExampleSlave()
    {
        Run += MutexExampleTask_Run;
    }

    void MutexExampleTask_Run(object sender, EventArgs e)
    {
        long id = Descriptor.Job.Id;

        string infoMessage = string.Format(
            "MutexExampleSlave starting. Job Id is {0}.", id);
        log.Info(infoMessage);

        /* Get the client id for logging purposes. */
        Guid clientId = new Guid(Descriptor.Job.CustomData.ToString());
        /* The GridSync dispatches calls to the web service,
         * and then on to the GridManager. */
        exampleSync = new GridSync(typeof(MutexExampleSlave), "test");

        StepsGoal = iterationsCount * 10;

        for (int i = 0; i < iterationsCount; i++)
        {
            /* Here we attempt to enter the monitor. 
             * This is done automatically when 
             * we instantiate a GridMonitor. 
             * The Monitor will Exit when it is disposed. 
             * We may block here for a while, 
             * and wait for another agent 
             * to leave the using statement. */
            using (new GridMonitor(exampleSync))
            {
                log.Info(string.Format(
                    "Locked id:{0} iteration:{1} cliendId: {2}", 
                    id, i, clientId));
                    
                for (int j = 1; j <= 10; j++)
                {
                    System.Threading.Thread.Sleep(200);
                    StepsCompleted = i * 10 + j;
                }
            }
            log.Info(string.Format(
                "Unlocked id:{0}, and now sleeping for 2 seconds.", id));
            System.Threading.Thread.Sleep(2000);
        }

        /* Once we leave this method, the Complete event will be raised. */
    }
}

The following excerpt from the GridMonitor class demonstrates how the Grid web service is called to relinquish its lock once it is disposed.

protected virtual void Dispose(bool IsDisposing)
{
    if (disposed)
    {
        return;
    }

    if (IsDisposing)
    {
        /* Allow other agents to enter using statement. */
        ReleaseLock();
    }

    /* Free any unmanaged resources 
     * in this section. */
    disposed = true;
}

The MutexExampleMaster task does very little appart from incrementing a task counter and dishing out ids for demonstration purposes at client-side. The code is provided here for good measure.

/// <summary>
/// This is the server side implementation for a task
/// that demonstrates the use of the GridMonitor.
/// </summary>
class MutexExampleMaster : MasterTask
{
    long taskCounter;
    const int runTimes = 100;

    public MutexExampleMaster()
    {
        StepsGoal = runTimes;
    }

    /// <summary>
    /// Gets the run data for the <see cref="Agent"/> slave task.
    /// This data should encapsulate the task segment
    /// that will be worked on by the slave. <seealso cref="Job"/>
    /// </summary>
    /// <param name="agent">The agent requesting the run data.</param>
    /// <returns>The job for the agent to work on.</returns>
    public override Job GetRunData(IAgent agent)
    {
        Job job = new Job(++taskCounter);
        job.CustomData = agent.Id;

        if (taskCounter >= runTimes)
        {
            throw new InvalidOperationException("Task is complete.");
        }
        StepsCompleted = taskCounter;

        return job;
    }

    /// <summary>
    /// Joins the specified task result. This is called
    /// when a slave task completes its <see cref="Job"/>,
    /// after having called <see cref="GetRunData"/>;
    /// returning the results to be integrated
    /// by the associated <see cref="MasterTask"/>.
    /// </summary>
    /// <param name="taskResult">The task result.</param>
    public override void Join(TaskResult taskResult)
    {
        if (taskCounter > 100)
        {
            Completed = true;
        }
    }
}

Expiring Collections

The most difficult challenge in creating a volunteer Grid computing platform that uses web clients is the volitility of the network, and the transient nature of the clients. At no time can we ever be certain that a node is still connected. When a client connects we assign it a connected window. This is a timespan in which the client must call home to be deemed still alive. If we are engaging in a concurrent activity we must retake ownership of any locks that the client has when the client does not call home within the window. Lest a deadlock may occur. Also, if we don't free up our data relating to a connection, then we will have a memory leak. We, therefore, periodically timeout Agents if they fail to call home. In order to accomplish this, I chose to implement a self contained mechanism for expiring client data. This comes in the form of two Expiring collections; an ExpiringDictionary and an ExpiringQueue. As it turned out, they proved to be very useful.

Figure: ExpiringDictionary and ExpiringQueue collections.

Both work the same as their compatriots in the System.Collections.Generic namespace, but with the added edition of a timer, and facility to touch items to renew their insertion date stamps. Both use an internal timer to eject items from their internal collections.

Figure: ExpiringDictionary. Items removed periodically by timer.

The ExpiringDictionary is used by the GridManager to keep track of connected Agents and their Jobs' progress, and also by the LockManager to associate locks with their respective Agents. The ExpiringQueue is also used by the LockManager, which maintains an ExpiringDictionary of LockSync and ExpiringQueue pairs. We associate a LockSync with a queue of Agents; thereby expiring all requests for a certain code block periodically, and also expiring individual requests for code blocks periodically. This minimises the amount of lock request information that we need to maintain. For example, if a MasterTask completes, then we can forget about all locks associated with its SlaveTask. Or if an user closes his or her browser, after having made a call to GridMonitor.Enter then we can safely remove the request, and free up the lock, after a specified time.

SlaveTasks are executed client-side on a low priority thread, and this is overseen by the TaskRunner.

Saving and retrieving data using the Isolated Store

Legion stores the number of tasks that have been processed by a user, using the Isolated Store. The Isolated Store is a virtual file system that is specific to each user. Thus, with multiple concurrent browsers for the same user, within the same domain, we are able to share data. I have provided a static IsolatedStorageUtility class to aid in storing and retrieving application settings and data. Unfortunately I was not able to provide the functionality that I had hoped because there isn't a Safe Critical facility to serialize objects as binary, XML, nor JSON. Once we can, perhaps in Silverlight 2.0, I will come back to it.

Figure: Silverlight Agent module, IsolatedStorage class diagram.

Saving data using the Isolated Store

using (IsolatedStorageFile storageFile
        = IsolatedStorageFile.GetUserStoreForApplication())
{
    if (value == null || value.ToString().Trim().Length == 0)
    {
        storageFile.DeleteFile(fileName);
    }
    else
    {
        using (IsolatedStorageFileStream isoStream
            = new IsolatedStorageFileStream(
                fileName, FileMode.Create, storageFile))
        {
            using (StreamWriter writer = new StreamWriter(isoStream))
            {
                writer.Write(value);
            }
        }
    }
}

Retrieving data using the Isolated Store

using (IsolatedStorageFile storageFile
    = IsolatedStorageFile.GetUserStoreForApplication())
{
    using (IsolatedStorageFileStream isoStream
        = new IsolatedStorageFileStream(
            fileName, FileMode.Open, storageFile))
    {
        using (StreamReader reader = new StreamReader(isoStream))
        {
            /* Read the to the end of the file. */
            String storedValue = reader.ReadToEnd();
            return storedValue;
        }
    }
}

How to write a Grid Task

So we've seen how Legion works internally, but what is probably of more interest to you, is how to create your own custom task. All we need to do is extend two classes, and add the task to the config. The two example tasks I have included with this release will provide you with the essentials.

First, create a new regular .NET class library project, and add a reference to the main GridComputing module. In it, create a new class that extends MasterTask, and provide overrides for the GetJob and Join methods. The first method GetJob is responsible for providing a client-side Agent with what it needs to run a task. The Join method takes the output from a SlaveTask and combines it with the output of other SlaveTasks.

Next, create a new Silverlight class library project, and add a reference to the SilverlightAgent project. This will contain the client-side code for the task. Create a new class and extend SlaveTask. Include a Post Build Event for the slave project, which will copy the built assembly to your website's ClientBin directory. It is not necessary to reference the Silverlight slave task project from the SilverlightAgent project. In fact, I would recommend that you do not. They must simply be copied to the websites ClientBin directory, and Silverlight will take care of downloading the assemblies to the browsers.

Figure: Post build event copies slave assembly to website Silverlight client bin.

Finally, define a new config element for the Task in the web.config of the website. This should include the fully qualified type name of the SlaveTask.

Future enhancements

• Provide a uniform shared storage facility for Agents.
• Implement half or full duplex channelling.

Conclusion

Web based Volunteer Grid computing offers a tremendous opportunity to harness the unused resources of visitors to a web site. We are able to leverage this resource in the form of a virtual super computer, and it affords us the opportunity to provide services in a manner outside of the traditional client-server model. This article discussed the design and use of Legion, a Grid Computing framework that uses the Silverlight CLR to execute user definable tasks. We saw how Legion uses an ASP.NET application and web services to download tasks, upload result data, and provide grid-wide thread-safe operations for agents. We also looked at how client performance metrics, such as bandwidth and processor speed, may be used to tailor the workload of Agents. It is my hope that this article will help to further interest in Grid computing, and also awareness of Silverlight as more than just a tool for UI.

I hope you find this project useful. If so, then I'd appreciate it if you would rate it and/or leave feedback below. This will help me to make my next article better.

Special Thanks

To Andre de Cavaignac of decav.com and LAB 49 for kindly allowing the use of his excellent WPF Line Graph.

References

• wpfedevcon, 2007, How to create reflections in Silverlight.
  Retrieved 20 December 2007 from MSDN
• Brown, P. 2007, Silverlight Assembly Preloader in Managed Code
  Retrieved 20 December 2007 from irritatedVowel.com
• Wikipedia, 2007, Grid computing
  Retrieved 20 December 2007 from Wikipedia.org
• pfisk, 2007, Speed Comparison of Smalltalk, IronPython, C# and ActionScript
  Retrieved 20 December 2007 from 21st Century Smalltalk
• Wikipedia, 2007, Smallpox
  Retrieved 19 December 2007 from 21st Wikipedia.org
• INQUIRER staff, 2003, US Army uses grid computing to crack smallpox virus
  Retrieved 19 December 2007 from the Inquirer

History

December 2007

• First release.

October 26 2008

• Support for Silverlight 2 RTM.

You must Sign In to use this message board.

FAQ   Search Messages    Layout Expand All MessagesThread ViewNo Javascript (slow)No Javascript Preview    Per page 102550

Msgs 1 to 25 of 100 (Total in Forum: 100) (Refresh) FirstPrevNext

How to add Tasks in the web.config? sover 5:53 4 Nov '08   Hi Daniel, Can you give me details for the required settings of the "type" and "slaveType" parameters of the "add name" section of the web.config file. For example, each of these appear to take a comma-delimited string. The "type" parameter appears to have two parts. I assume the first part is namespace.classname of the master task and the second part is the name of master assembly. Is this correct? If not, please enlighten me. I have tried to include my custom task using these assumptions, but I get this runtime error when the project first loads: {"The value of the property 'type' cannot be parsed. The error is: The type 'Orpius.GridComputing.Tasks.MyTaskMaster, Orpius.GridComputing.Tasks.MyTask' cannot be resolved. Please verify the spelling is correct or that the full type name is provided. (D:\\Proj\\Legion\\Source\\GridComputing\\Website\\web.config line 33)"} Thanks for a great article! Steven Sign In·View Thread·PermaLink Re: How to add Tasks in the web.config? Daniel Vaughan 13:59 4 Nov '08   Steven, The format, as you described it, is correct. You can resolve this in two ways. Either add a reference to your assembly Orpius.GridComputing.Tasks.MyTask in the website project, or set the output directory of your master task project to be the bin directory of the website. Thanks for your message and kind words. Cheers, Daniel Daniel Vaughan Blog: DanielVaughan.Orpius.com Sign In·View Thread·PermaLink [Message Removed] Katekortez 10:07 25 Oct '08   Spam message removed Sign In·View Thread·PermaLink Advice for anyone trying to open project in VS2008 [modified] EricFromMars 14:34 27 Aug '08   Great article! Congratulations on an excellent article. Your presentation was excellent and, after reading it, I downloaded the source to have a look. When I tried to open the solution in VS2008 (Team System with SP1 installed) I received the following error: The project file 'x\x\x\x\Legion\Source\GridComputing\SilverlightAgent\SilverlightAgent.csproj' cannot be opened. The project type is not supported by this installation. The issue was caused by me not having Microsoft Silverlight 1.1 Tools Alpha for Visual Studio 2008 installed. Oh, and if you receive the following error you've installed the Beta Tools, not the Alpha Package 'Microsoft.VisualStudio.Silverlight.SLPackage, Microsoft.VisualStudio.Silverlight, Version=9.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' has failed to load properly ( GUID = {CB22EE0E-4072-4AE7-96E2-90FCCF879544} ). Please contact package vendor for assistance. Application restart is recommended, due to possible environment corruption. Would you like to disable loading this package in the future? You may use 'devenv /resetskippkgs' to re-enable package loading. modified on Wednesday, August 27, 2008 7:45 PM Sign In·View Thread·PermaLink Re: Advice for anyone trying to open project in VS2008 Daniel Vaughan 17:50 27 Aug '08   Yes, the current source is for Silverlight 1.1. I will be releasing a new version for Silverlight 2, and I have almost completed the conversion to Silverlight 2 beta 2, however some configuration challenges still remain. Thanks for your comment. I’m glad you liked the article. Cheers, Daniel Daniel Vaughan Blog: DanielVaughan.Orpius.com Sign In·View Thread·PermaLink Re: Advice for anyone trying to open project in VS2008 Member 807123 5:50 3 Sep '08   Hello Daniel, Have you got a rough estimate as to when the Silverlight Beta 2 compatible version will be available? Cheers, Jonathan. Sign In·View Thread·PermaLink Re: Advice for anyone trying to open project in VS2008 Daniel Vaughan 6:10 27 Oct '08   If you're still interested I've updated the codebase for Silverlight 2 RTM. Cheers, Daniel Daniel Vaughan Blog: DanielVaughan.Orpius.com Sign In·View Thread·PermaLink SlaveTask's dll's problem Aleksey Zaharov 10:02 17 Aug '08   Hi Daniel, In silverlight 1.1 it was possible to copy slave task's dll to client bin on sucess build event. But in Silverlight 2.0 we have xap package.. and copying of dll's no longer works. Do you know how to solve this problem? Thx, Alexey. Sign In·View Thread·PermaLink Re: SlaveTask's dll's problem Daniel Vaughan 15:37 21 Aug '08   Copying assemblies can be achieved by setting the output directory in the project properties. But, I believe your real question may be in regards to automatic resolution of types in the client application. In my upcoming release I have solved this problem by manually downloading assemblies. Cheers, Daniel Daniel Vaughan Blog: DanielVaughan.Orpius.com Sign In·View Thread·PermaLink Re: SlaveTask's dll's problem Aleksey Zaharov 22:32 22 Aug '08