Click here to Skip to main content
15,867,568 members
Articles / Programming Languages / C#

Grid Computing Using C# Script and .NET Remoting

Rate me:
Please Sign up or sign in to vote.
5.00/5 (20 votes)
10 Apr 2010CPOL6 min read 74.7K   2K   59   24
Using the C# script engine inside a network using .NET Remoting.

Introduction

If I count the number of top articles I have seen till now, the article "C# Script: The Missing Puzzle Piece" by "Oleg Shilo" is one of them. If you didn't see that I recommended you to view it, then come back to this article; because this article is based on the C# script engine.

C# script is a solution for making C# code act as a script language. It uses the C# compiling engine for compilation and then executes it. 

When I saw that article, I thought what would happen if I could use this engine on the network? And I started making a solution to use it on several computers that were connected together.

I don't want to describe the C# Script Engine here. For more information, see the main article or the CS-Script home page.

Usages

We can use this idea to compute long-running algorithms on more than one computer by paralleling the execution (called "Grid Computing"). For more information, see the sample at the end of this article that I use to calculate the pi number using my solution.

Another great usage of C# script on network is executing some actions and jobs on the network, like remote management and everything else you can imagine!

Just think a little about the uses of this idea and you will find this idea very useful!

Security Attention

When you use this project, be aware that it can be a back door for computers where the CS-Script Service have been installed on them (our executers) because it can execute any code that is requested!

For simplicity, I didn't add authentication and encryption techniques to authorize the execution requester, but I'm going to add these policies to the solution later, and I'll be glad to hear your suggestions. If you want to implement some security options for .NET Remoting, you can read the article on MSDN about "Writing an Asymmetric Encryption Channel Sink".

.NET Remoting

As you may know, .NET Remoting is a technology of .NET for communication of processes over the network. If you are not familiar with .NET Remoting, you can read articles about it like this and this. I am using .NET Remoting for communication between a manager and the executers (both are described below). For getting away from firewalls and security boundaries, I used the HTTP channel and the SOAP formatter.

Introduction to Grid Computing

One of the main strategies of grid computing is using software to divide and apportion pieces of a program among several computers, sometimes up to many thousands. Grid computing is distributed, large-scale cluster computing, as well as a form of network-distributed parallel processing [1].

I assumed a simple shape of grid computing with one manager and some executers connected to the manager. 

MEEEE.png

We don't have any restrictions on the number of executers. They can grow up as much as till the resources let us.

Solution Contents

There are several projects in the solution, and I'm going to describe each of them below:

  • RemoteClass: Contains a class and an interface that both clients and server use as a remote object for invoking methods and returning results.
  • CSScriptService: This is a Windows Service program that plays our executer rule.
  • ExecutionManager: A sample manager that sends code to an executer, and then the executer compiles the code and executes it and gives back the results to the caller (Manager).
  • ClientTest and ServerTest: If you don't want to install the service for testing the system, these two projects are for you. Just run the server and then the client.
  • PiCalculationManager: A sample program for grid computing with the CS-Script Service; plays the role of the manager, just like ExecutionManager.

1. RemoteClass

This is a class library that consists of a collection of classes and interfaces; both the manager and executers reference it.

C#
public interface IRemoteObject
{
    [OneWay]
    void ExecuteCode(string code, string method, string[] parameters);
    [OneWay]
    void ExecuteAlreadyExecuted(string hash, string method, string[] parameters);
    [OneWay]
    void ExecuteAssembly(byte[] bytes, string method, string[] parameters);

    void RegisterCallback(ReturnEvent delCallback);
    bool Echo();
}

IRemoteObject is an interface that describes the services which the CS-Script Engine provides. The OneWay attribute is an attribute of the System.Runtime.Remoting.Messaging namespace that makes a method of remote object as one way, without a return value, and the caller does not need to wait to finish execution of the method on the server side (here, executers), and returns immediately. It means that methods invocation is asynchronous; the manager calls the method and forgets its invocation; after finishing the method, the executer invokes a manager's method and sends the results.

There are five methods on it described below:

  • RegisterCallback: The manager sends an instance of the ReturnEvent delegate to the executer that points to the method that will be invoked when the executer finishes its work.
  • ExecuteCode: Gives a C# code as a string, the method on the code that is executed, and its parameters. Then, the CS-Script Engine compiles the code and runs the specified method of the created assembly.
  • ExecuteAssembly: You can send an assembly directly instead of its code to run. It's just like the ExecuteCode method.
  • ExecuteAlreadyExecuted: After executing a code or an assembly by one of the two methods above, you don't need to resend that; just specify the hash code that exists in the first execution result. This idea improves the execution performance, especially in grid computing processes that need to execute some methods periodically.
  • Echo: This is just a test method to find out if a remote executer works or not.
RemoteClass.cs
C#
public sealed class RemoteClass : MarshalByRefObject, IRemoteObject
{
        private static Dictionary<string, string> executedFiles = 
                new Dictionary<string, string>();
        private ReturnEvent delCallback;
        private Assembly lastAssembly;
        private string lastCodeHash;

        
        public void ExecuteCode(string code, string method, string[] parameters)
        {
            string hash = ComputeHash(code);
            if (executedFiles.ContainsKey(hash) && File.Exists(executedFiles[hash]))
            {
                ExecuteAlreadyExecuted(hash, method, parameters);
                return;
            }
            try
            {
                Assembly asm = CSScript.LoadCode(code, null, false);
                Execute(asm, hash, method, parameters);
            }
            catch (Exception ex)
            {
                ReturnEventArgs e = new ReturnEventArgs();
                e.ExecuterException = ex;
                if (delCallback != null)
                    delCallback(e);
            }
        }

        public void ExecuteAlreadyExecuted(string hash, string method, string[] parameters)
        {
            if (!executedFiles.ContainsKey(hash) || !File.Exists(executedFiles[hash]))
            {
                ReturnEventArgs e = new ReturnEventArgs();
                e.ExecuterException = new FileNotFoundException();
                if (delCallback != null)
                    delCallback(e);
                return;
            }
            try
            {
                Assembly asm;
                if (hash == lastCodeHash && lastAssembly != null)
                {
                    asm = lastAssembly;
                }
                else
                {
                    asm = Assembly.LoadFrom(executedFiles[hash]);
                }
                Execute(asm, hash, method, parameters);
            }
            catch (Exception ex)
            {
                ReturnEventArgs e = new ReturnEventArgs();
                e.ExecuterException = ex;
                if (delCallback != null)
                    delCallback(e);
            }

        }

        public void ExecuteAssembly(byte[] bytes, string method, string[] parameters)
        {
            string commonCacheDir = Path.Combine(GetScriptTempDir(), "Cache");
            if (!Directory.Exists(commonCacheDir))
                Directory.CreateDirectory(commonCacheDir);
            string tempFile = Path.Combine(commonCacheDir, 
                   commonCacheDir.GetHashCode().ToString() + ".dll");
            File.WriteAllBytes(tempFile, bytes);
            try
            {
                string hash = ComputeHash(bytes);
                Assembly asm = Assembly.LoadFrom(tempFile);
                Execute(asm, hash, method, parameters);
            }
            catch (Exception ex)
            {
                ReturnEventArgs e = new ReturnEventArgs();
                e.CodeException = ex;
                if (delCallback != null)
                    delCallback(e);
            }
        }

        public static string GetScriptTempDir()
        {
            string dir = Path.Combine(Path.GetTempPath(), "CSSCRIPT");
            if (!Directory.Exists(dir))
                Directory.CreateDirectory(dir);
            return dir;
        }

        private void Execute(Assembly asm, string hash, string method, string[] parameters)
        {
            lastCodeHash = hash;
            lastAssembly = asm;
            executedFiles[hash] = asm.Location;
            ExecutionParameter p = new ExecutionParameter();
            p.Method = method;
            p.Parameters = parameters;
            p.E.HashCode = hash;
            p.Assembly = asm;
            Thread th = new Thread(new ParameterizedThreadStart(Execute));
            th.Start(p);
        }

        private void Execute(object p)
        {
            if (!(p is ExecutionParameter))
                throw new InvalidCastException();
            ExecutionParameter param = (ExecutionParameter)p;
            try
            {
                object retObj = param.Assembly.GetHelper().Invoke(
                                param.Method, new object[] { param.Parameters });
                if (retObj != null)
                    param.E.CustomReturnValue = retObj.ToString();
            }
            catch (Exception ex)
            {
                param.E.CodeException = ex;
            }
            if (delCallback != null)
                delCallback(param.E);
        }

        public void RegisterCallback(ReturnEvent delCallback)
        {
            this.delCallback = delCallback;
        }

        public bool Echo()
        {
            return true;
        }

        public static string ComputeHash(string str)
        {
            return ComputeHash(Encoding.Unicode.GetBytes(str));
        }

        public static string ComputeHash(byte[] bytes)
        {
            MD5 alg = MD5CryptoServiceProvider.Create();
            byte[] res = alg.ComputeHash(bytes);
            return KeyEncoder.GetString(res);
        }
}

As you can see, RemoteClass implements the IRemoteObject class and inherits from the MarshalByRefObject class; because each class that wants to be a remote class must inherit from MarshalByRefObject.

2. CS-Script Service

This is our executer program in the face of the Windows service. I used code from the article "Windows Services Can Install Themselves" to enable the service to install itself.

CSScriptService.cs
C#
public partial class CSScriptService : ServiceBase
{
    private IChannel channel;

    public CSScriptService()
    {
        InitializeComponent();
    }

    protected override void OnStart(string[] args)
    {
        Hashtable props = new Hashtable();
        props["port"] = 8086;
        SoapServerFormatterSinkProvider serverProv = 
          new SoapServerFormatterSinkProvider();
        serverProv.TypeFilterLevel = 
          System.Runtime.Serialization.Formatters.TypeFilterLevel.Full;
        channel = new HttpChannel(props, null, serverProv);
        ChannelServices.RegisterChannel(channel, false);
        RemotingConfiguration.RegisterWellKnownServiceType(typeof(RemoteClass), 
                              "Execute", WellKnownObjectMode.Singleton);
    }

    protected override void OnStop()
    {
        ChannelServices.UnregisterChannel(channel);
        try
        {
            Directory.Delete(RemoteClass.GetScriptTempDir(), true);
        }
        catch { }
    }

    protected override void OnPause()
    {
        ChannelServices.UnregisterChannel(channel);
    }

    protected override void OnContinue()
    {
        ChannelServices.RegisterChannel(channel, false);
    }
}

2.1. Installing CS-Script Service

To install the executer service, copy the CSScriptService release directory to a specified directory (e.g., C:\Program files\MyExecuter), then in the command prompt, run: "CSScriptService.exe -“i" to install the service.

For starting the service and to set some options, open Administrative Tools -> Services and then find CSScriptService. Open the properties window for this service. On the "Log On" tab, check the "Allow the service to interact with the desktop" check box. This check box allows the executer to run codes that need interaction with the user, for example, show Windows Forms or a message box. After setting the "Log On" tab, get back to the "General" tab and start the service.

image002.png

image003.png

3. ExecutionManager

image004.png

This is a sample manager that sends code to an executer, then the executer compiles the code and executes it and gives back its results to the caller (Manager). Don't forget that the specified method must be static, and both the method and its related class must be public.

A Sample of Grid Computing: Pi Number Calculation

Here, I have found an algorithm for calculating the digits of the pi number using an algorithm of "Fabrice Bellard". The algorithm has a method that gives an integer as an argument and returns a string containing 9 digits of the pi number digits from a given integer.

I made a manager like the manager described above, but it gives an array if IP addresses of executers, and after clicking the "Start" button, it sends the code for the pi calculation algorithm to each of them and requests to calculate some pieces of the pi number.

image005.png

This is a chart that shows a comparison of the time taken to calculate 1000 digits of the pi number in different numbers of similar executers in a network:

image006r.PNG

References

  1. Fran Berman, Geoffrey Fox, Anthony J.G. Hey, Grid Computing: Making the Global Infrastructure a Reality. Wiley, 2008.

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)


Written By
Software Developer
Iran (Islamic Republic of) Iran (Islamic Republic of)
I'm an undergraduate student of IT.
I'm a C# programmer and have some experience in robotics, image processing, telephony systems, etc.

Comments and Discussions

 
QuestionAny idea about... Pin
Member 1213277915-Feb-17 5:47
Member 1213277915-Feb-17 5:47 
Questionproject doesn't build at all after one way upgrade to visual studio 2013 Pin
Franklin Moormann20-Jan-15 10:11
Franklin Moormann20-Jan-15 10:11 
Questionschedulers+grids Pin
Denry0n.27-Jan-14 4:23
Denry0n.27-Jan-14 4:23 
Questionproblem :( Pin
mehdi nasrollahpour27-Dec-13 8:33
mehdi nasrollahpour27-Dec-13 8:33 
QuestionMatrix Determinant Calculation Pin
Morteza Giti13-Dec-13 3:43
Morteza Giti13-Dec-13 3:43 
GeneralMy vote of 5 Pin
Mic13-Nov-12 4:41
Mic13-Nov-12 4:41 
Questionhiya there Pin
P. Gopinathan18-Jan-12 0:35
P. Gopinathan18-Jan-12 0:35 
AnswerRe: hiya there Pin
Ali Aboutalebi18-Jan-12 7:18
Ali Aboutalebi18-Jan-12 7:18 
GeneralRe: hiya there Pin
P. Gopinathan18-Jan-12 17:33
P. Gopinathan18-Jan-12 17:33 
Sorry to disturb you sir. . . I converted that file to .net 2010 in that i don't know how to start the CSScriptService.exe. . . I can't find those exe file. . . I'm new to this type of programs that's why I'm asking you. . .
GeneralRe: hiya there Pin
Ali Aboutalebi18-Jan-12 22:54
Ali Aboutalebi18-Jan-12 22:54 
GeneralRe: hiya there Pin
P. Gopinathan26-Jan-12 23:06
P. Gopinathan26-Jan-12 23:06 
Generalrepeatedly work Pin
A. Najafzadeh27-May-11 9:30
A. Najafzadeh27-May-11 9:30 
GeneralCan not connect to other PC Pin
Nguyen Thai Hoang20-Apr-11 0:00
Nguyen Thai Hoang20-Apr-11 0:00 
GeneralRe: Can not connect to other PC Pin
Ali Aboutalebi20-Apr-11 6:38
Ali Aboutalebi20-Apr-11 6:38 
GeneralRe: Can not connect to other PC [modified] Pin
Nguyen Thai Hoang21-Apr-11 5:51
Nguyen Thai Hoang21-Apr-11 5:51 
GeneralRe: Can not connect to other PC Pin
Ali Aboutalebi24-Apr-11 10:02
Ali Aboutalebi24-Apr-11 10:02 
GeneralRe: Can not connect to other PC Pin
Nguyen Thai Hoang25-Apr-11 0:30
Nguyen Thai Hoang25-Apr-11 0:30 
GeneralRe: Can not connect to other PC Pin
Ali Aboutalebi25-Apr-11 1:28
Ali Aboutalebi25-Apr-11 1:28 
GeneralRe: Can not connect to other PC Pin
Luxor835-Nov-14 21:59
Luxor835-Nov-14 21:59 
General5 for 5 Pin
kdgupta877-Jan-11 1:40
kdgupta877-Jan-11 1:40 
GeneralRe: 5 for 5 Pin
Ali Aboutalebi20-Apr-11 6:39
Ali Aboutalebi20-Apr-11 6:39 
Jokeoff topic. Pin
Diamonddrake13-Apr-10 13:35
Diamonddrake13-Apr-10 13:35 
GeneralAddressing security cipncerns Pin
Oleg Shilo10-Apr-10 15:41
Oleg Shilo10-Apr-10 15:41 
GeneralRe: Addressing security cipncerns Pin
Ali Aboutalebi10-Apr-10 20:02
Ali Aboutalebi10-Apr-10 20:02 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.