I'm building a Windows Service that takes care of our ETL process by executing packages and SQL commands on different cues, like files dropped in a folder and such. This must be done in a windows service hosted on our data warehouse server, because it needs to be monitored from different computers by different users.
I've read about the ServiceController but that provides very little functionality. I have created a library of objects that take care of execution sometimes in parallel and sometimes sequentially. These objects are contained in one main object (like a tree with nodes, where each node contains more nodes, but all have the same root). Each object has a different state and I must monitor these states and all the instances of the client application must reflect the same changes.
I learned I can have a more verbose communication between my service and the client application using WCF. The way I plan it, the client application will poll the service at a certain time interval and the service will send the root object that I can work with (view their states).
The problem is that I sometimes need to stop the execution of certain objects, or just call other methods without stopping the entire service. I quickly learned that the instances that implement the service contract at client-side are marshal by value, thus changing them won't be reflected at the server-side. I read about the EndpointAddress10 but that requires a different instance of the server object for each client instance. What I need is, if it is possible, that all client applications work with the same instance at server-side. There won't be any problems with racing, since the guys monitoring it just take turns over the night and during the day, and in a perfect scenario (no errors in the ETL) interaction won't be needed. This is mostly for trouble shooting and changing configuration settings (which won't happen often).
Is what I'm trying to do possible? Different users working with the same instance of the process, receiving data and calling methods to be run at server-side but logged remotely from their own work stations. What is the best way to do this? My client-side application is pretty heavy.
UPDATE:
I'll try to explain in more detail. I have a tree of objects that can each be executed, and each has a state (Pending, Running, Succeded, Failed, etc.). The ETL process is unique. Some CSV files arrive at a specified folder and the packages are triggered accordingly. After this, there's a staging phase where I'm not loading files anymore but just running SQL statements and packages that process the info. The objects I created take care of the order in which each should be executed (there are sequences and parallel executions). Sometimes, when one package fails the others that depend on its success are canceled. When the operator fixes the problem, he should be able to restart the process where it left off.
Right now this is all done in a windows forms app. This has the disadvantage of running in a specific user's session, thus only said user can view the progress and intereact with it. If a different user wanted to see the progress he wouldn't be able to because the application isn't running in his session. The idea is that a service domain account will be used to run the windows service and all users will use the windows forms application to communicate with the service. I need to be able to restart execution and call other methods from the client applications but everything will be executed server side. The client side is just a proxy.
Submit an article or tip