Click here to Skip to main content
Click here to Skip to main content
Go to top

Topshelf’s Windows Service and try-finally

, 7 Dec 2012
Rate this:
Please Sign up or sign in to vote.
I recently needed to diagnose an interesting problem with one of our Topshelf’s Windows Services. The service ended in a StopPending state and we needed to kill it in order to make it work again. But before killing the service … Continue reading →

I recently needed to diagnose an interesting problem with one of our Topshelf’s Windows Services. The service ended in a StopPending state and we needed to kill it in order to make it work again. But before killing the service we collected a process dump for further analysis. In this post I will show you what I read from this dump and how this information might help you better understand your own Topshelf applications.

The aforementioned service is a multithreaded application with each thread processing messages from a queue. I prepared a simplified version of it for the purpose of this post:

namespace LowLevelDesign.Samples
{
    class TestWorker : ServiceControl
    {
        private const int ThreadCount = 5;
        private static readonly LogWriter logger = HostLogger.Get<TestWorker>();
        public static bool ShouldStop { get; private set; }
        private WaitHandle[] handles;
        
        public bool Start(HostControl hostControl)
        {
            logger.Info("Starting test worker...");

            handles = new ManualResetEvent[ThreadCount];
            for (int i = 0; i < handles.Length; i++)
            {
                handles[i] = new ManualResetEvent(false);
            }
            logger.Info("Starting worker threads...");
            for (int i = 0; i < ThreadCount; i++)
            {
                // start the listenening thread
                ThreadPool.QueueUserWorkItem(Test, handles[i]);
            }
            return true;
        }
        
        private static void Test(Object state) 
        {
            var h = (ManualResetEvent)state;
            try 
            {
                logger.InfoFormat("Throwing exception");
                throw new Exception();
            }
            finally 
            {
                logger.InfoFormat("Releasing the handle");
                h.Set();
            }
        }

        public bool Stop(HostControl hostControl)
        {
            ShouldStop = true;
            logger.Info("Stopping test worker...");
            // wait for all threads to finish
            WaitHandle.WaitAll(handles);

            return true;
        }
    }

    class Program
    {
        static void Main()
        {
            HostFactory.Run(hc =>
            {
                hc.UseNLog();
                // service is constructed using its default constructor
                hc.Service<TestWorker>();
                // sets service properties
                hc.SetServiceName(typeof(TestWorker).Namespace);
                hc.SetDisplayName(typeof(TestWorker).Namespace);
                hc.SetDescription("Test worker");
            });
        }
    }
}

As you can read from the snippet, we create 5 threads on the service start (through ThreadPool), assigning each of them an instance of ManualResetEvent. Each thread throws an exception which forces the service to stop. I expected the service to wait for all threads to finish and then exit. Unfortunately the reality was surprisingly different and my service became unresponsive after a while.

Let’s look at the Topshelf source code in order to find out what’s going on. When Topshelf starts a service it subscribes to all the unhandled exceptions that might happen during the service execution:

namespace Topshelf.Runtime.Windows
{
    public class WindowsServiceHost :
        ServiceBase,
        Host,
        HostControl
    {
        ...
        public TopshelfExitCode Run()
        {
            Directory.SetCurrentDirectory(AppDomain.CurrentDomain.BaseDirectory);

            AppDomain.CurrentDomain.UnhandledException += CatchUnhandledException;
            ...
            Run(this);
            ...
       }
       ...
    }
}

Then our TestWorker.Start method is called which fires 5 threads. For each thrown exception the Topshelf’s WindowsServiceHost.CatchUnhandledException method is called:

namespace Topshelf.Runtime.Windows
{
    public class WindowsServiceHost :
        ServiceBase,
        Host,
        HostControl
    {
        ...
        void CatchUnhandledException(object sender, UnhandledExceptionEventArgs e)
        {
            _log.Error("The service threw an unhandled exception", (Exception)e.ExceptionObject);

            Stop();


#if !NET35
            // it isn't likely that a TPL thread should land here, but if it does let's no block it
            if (Task.CurrentId.HasValue)
            {
                return;
            }
#endif

            int deadThreadId = Interlocked.Increment(ref _deadThread);
            Thread.CurrentThread.IsBackground = true;
            Thread.CurrentThread.Name = "Unhandled Exception " + deadThreadId.ToString();
            while (true)
                Thread.Sleep(TimeSpan.FromHours(1));
        }
        ...
    }
}

Notice that each unhandled exception calls ServiceBase.Stop method (which eventually runs our TestWorker.Stop method) and finally gets into an infinite loop. I guess you now know why our simple service could not be stopped – the finally {} block in a thread code was never called thus keeping the thread’s ManualResetEvent in an unsignaled state and effectively blocking the TestWorker.Stop from finishing.

A simple resolution to this deadlock situation might be adding a catch {} block that will signal the ManualResetEvent before handling execution to WindowsServiceHost.CatchUnhandledException:

        private static void Test(Object state) 
        {
            var h = (ManualResetEvent)state;
            try 
            {
                throw new Exception();
            } 
            /* the missing part: */
            catch 
            {
                logger.InfoFormat("Releasing the handle on exception.");
                h.Set();
                throw;
            }
            finally 
            {
                logger.InfoFormat("Releasing the handle");
                h.Set();
            }
        }

If you have any other ideas how to fix this problem please write them in comments. And if you would like to experiment with the code presented in this post, it’s available for download here.


Filed under: CodeProject, Diagnosing Windows Services

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Share

About the Author

Sebastian Solnica
Software Developer (Senior)
Poland Poland
Interested in tracing, debugging and performance tuning of the .NET applications (especially ASP.NET).
 
If you find this article interesting, maybe you would like to pay me a visit: http://lowleveldesign.wordpress.com? Smile | :)

Comments and Discussions

 
-- There are no messages in this forum --
| Advertise | Privacy | Mobile
Web03 | 2.8.140921.1 | Last Updated 7 Dec 2012
Article Copyright 2012 by Sebastian Solnica
Everything else Copyright © CodeProject, 1999-2014
Terms of Service
Layout: fixed | fluid