Service Monitor Application with .NET Core

• GitHub Repository

Introduction

This guide is about how to create a service monitor application, but what is it? In simple words: it's an application that allows to monitor services in a network and save results from monitoring in a database, SQL Server for this case.

I know there are a lot of tools that can provide this feature, also there are better tools that money can buy but my intention with this guide is to show how to use .NET Core power to build an application that developers can extend for custom requirements.

The basic idea is this: have a process to run in infinite ways to monitor hosts, databases and APIs; save monitoring results in SQL Server database, then we can build a fancy UI to end-user and show status for each service, we can have a lot of targets to monitoring but it's better to allow users to subscribe for specific services and not all; for example DBAs need to watch database servers not APIs, developers need to watch development databases and APIs, etc.

Also think about having big monitors in your development room and watching the status for your services and in the best of cases, have charts. :)

One special feature could be to have a notification service to send messages for all administrators in case one or more services fail, in this context, service means a target such as host, database, API.

In this guide, we'll work with monitoring the following services:

Background

As we said before, we'll create an application to monitoring existing targets (hosts, databases, APIs), so we need to have basic knowledge about these concepts.

Hosts will be monitoring with ping action, so we'll add networking related packages to perform this action.

Databases will be monitoring with open and close connections, don't use integrated security because you'll need to impersonate your service monitor process with your credentials, so in that case, it's better to have a specific user to connect with database and only that action to avoid hacking.

RESTful APIs will be monitoring with REST client to target an action that returns simple JSON.

Database

Inside of repository, there is a directory with name \Resources\Database and this directory contains related database files, please make sure to run the following files in this order:

We can found scripts for database here.

.NET Core Solution

Now we need to define the projects for this solution to get a clear concept about project's scope:

ServiceMonitor.Core

This project contains all definitions for entities and database access, so we need to add the following packages for project:

This project contains three layers: Business Logic, Database Access and Entities; please take a look at the article EF Core for Entreprise to get a better understanding about this project and layers.

DashboardService class code:

using System;
using System.Threading.Tasks;
using Microsoft.EntityFrameworkCore;
using Microsoft.Extensions.Logging;
using ServiceMonitor.Core.BusinessLayer.Contracts;
using ServiceMonitor.Core.BusinessLayer.Responses;
using ServiceMonitor.Core.DataLayer;
using ServiceMonitor.Core.DataLayer.DataContracts;
using ServiceMonitor.Core.EntityLayer;

namespace ServiceMonitor.Core.BusinessLayer
{
    public class DashboardService : Service, IDashboardService
    {
        public DashboardService(ILogger<DashboardService> logger, ServiceMonitorDbContext dbContext)
            : base(logger, dbContext)
        {
        }

        public async Task<IListResponse<ServiceWatcherItemDto>> GetActiveServiceWatcherItemsAsync()
        {
            Logger?.LogDebug("'{0}' has been invoked", nameof(GetActiveServiceWatcherItemsAsync));

            var response = new ListResponse<ServiceWatcherItemDto>();

            try
            {
                response.Model = await DbContext.GetActiveServiceWatcherItems().ToListAsync();

                Logger?.LogInformation("The service watch items were loaded successfully");
            }
            catch (Exception ex)
            {
                response.SetError(Logger, nameof(GetActiveServiceWatcherItemsAsync), ex);
            }

            return response;
        }

        public async Task<IListResponse<ServiceStatusDetailDto>> GetServiceStatusesAsync(string userName)
        {
            Logger?.LogDebug("'{0}' has been invoked", nameof(GetServiceStatusesAsync));

            var response = new ListResponse<ServiceStatusDetailDto>();

            try
            {
                var user = await DbContext.GetUserAsync(userName);

                if (user == null)
                {
                    Logger?.LogInformation("There isn't data for user '{0}'", userName);

                    return new ListResponse<ServiceStatusDetailDto>();
                }
                else
                {
                    response.Model = await DbContext.GetServiceStatuses(user).ToListAsync();

                    Logger?.LogInformation("The service status details for '{0}' user were loaded successfully", userName);
                }
            }
            catch (Exception ex)
            {
                response.SetError(Logger, nameof(GetServiceStatusesAsync), ex);
            }

            return response;
        }

        public async Task<ISingleResponse<ServiceEnvironmentStatus>> GetServiceStatusAsync(ServiceEnvironmentStatus entity)
        {
            Logger?.LogDebug("'{0}' has been invoked", nameof(GetServiceStatusAsync));

            var response = new SingleResponse<ServiceEnvironmentStatus>();

            try
            {
                response.Model = await DbContext.GetServiceEnvironmentStatusAsync(entity);
            }
            catch (Exception ex)
            {
                response.SetError(Logger, nameof(GetServiceStatusAsync), ex);
            }

            return response;
        }
    }
}

ServiceMonitor.Common

Contracts

• IWatcher
• IWatchResponse
• ISerializer

IWatcher interface code:

using System.Threading.Tasks;

namespace ServiceMonitor.Common.Contracts
{
    public interface IWatcher
    {
        string ActionName { get; }

        Task<WatchResponse> WatchAsync(WatcherParameter parameter);
    }
}

IWatchResponse interface code:

namespace ServiceMonitor.Common.Contracts
{
    public interface IWatchResponse
    {
        bool Success { get; set; }

        string Message { get; set; }

        string StackTrace { get; set; }
    }
}

ISerializer interface code:

namespace ServiceMonitor.Common.Contracts
{
    public interface ISerializer
    {
        string Serialize<T>(T obj);

        T Deserialze<T>(string source);
    }
}

Watchers

These are the implementations:

• DatabaseWatcher
• HttpRequestWatcher
• PingWatcher

DatabaseWatcher class code:

using System;
using System.Data.SqlClient;
using System.Threading.Tasks;
using ServiceMonitor.Common.Contracts;

namespace ServiceMonitor.Common
{
    public class DatabaseWatcher : IWatcher
    {
        public string ActionName
            => "OpenDatabaseConnection";

        public async Task<WatchResponse> WatchAsync(WatcherParameter parameter)
        {
            var response = new WatchResponse();

            using (var connection = new SqlConnection(parameter.Values["ConnectionString"]))
            {
                try
                {
                    await connection.OpenAsync();

                    response.Success = true;
                }
                catch (Exception ex)
                {
                    response.Success = false;
                    response.Message = ex.Message;
                    response.StackTrace = ex.ToString();
                }
            }

            return response;
        }
    }
}

HttpWebRequestWatcher class code:

using System;
using System.Threading.Tasks;
using ServiceMonitor.Common.Contracts;

namespace ServiceMonitor.Common
{
    public class HttpRequestWatcher : IWatcher
    {
        public string ActionName
            => "HttpRequest";

        public async Task<WatchResponse> WatchAsync(WatcherParameter parameter)
        {
            var response = new WatchResponse();

            try
            {
                var restClient = new RestClient();

                await restClient.GetAsync(parameter.Values["Url"]);

                response.Success = true;
            }
            catch (Exception ex)
            {
                response.Success = false;
                response.Message = ex.Message;
                response.StackTrace = ex.ToString();
            }

            return response;
        }
    }
}

PingWatcher class code:

using System.Net.NetworkInformation;
using System.Threading.Tasks;
using ServiceMonitor.Common.Contracts;

namespace ServiceMonitor.Common
{
    public class PingWatcher : IWatcher
    {
        public string ActionName
            => "Ping";

        public async Task<WatchResponse> WatchAsync(WatcherParameter parameter)
        {
            var ping = new Ping();

            var reply = await ping.SendPingAsync(parameter.Values["Address"]);

            return new WatchResponse
            {
                Success = reply.Status == IPStatus.Success ? true : false
            };
        }
    }
}

ServiceMonitor.WebAPI

This project represents RESTful API for service monitor, so We'll have two controllers: DashboardController and AdministrationController. Dashboard has all operations related to end user results and Administration contains all operations related to save information (create, edit and delete).

Dashboard

DashboardController class code:

using System.Threading.Tasks;
using Microsoft.AspNetCore.Mvc;
using Microsoft.Extensions.Logging;
using ServiceMonitor.Core.BusinessLayer.Contracts;
using ServiceMonitor.WebAPI.Responses;

namespace ServiceMonitor.WebAPI.Controllers
{
#pragma warning disable CS1591
    [Route("api/v1/[controller]")]
    [ApiController]
    public class DashboardController : ControllerBase
    {
        protected readonly ILogger Logger;
        protected readonly IDashboardService Service;

        public DashboardController(ILogger<DashboardController> logger, IDashboardService service)
        {
            Logger = logger;
            Service = service;
        }
#pragma warning restore CS1591

        /// <summary>
        /// Gets service watcher items (registered services to watch with service monitor)
        /// </summary>
        /// <returns>A sequence of services to watch</returns>
        /// <response code="200"></response>
        /// <response code="500"></response>
        [HttpGet("ServiceWatcherItem")]
        [ProducesResponseType(200)]
        [ProducesResponseType(500)]
        public async Task<IActionResult> GetServiceWatcherItemsAsync()
        {
            Logger?.LogDebug("'{0}' has been invoked", nameof(GetServiceWatcherItemsAsync));

            var response = await Service.GetActiveServiceWatcherItemsAsync();

            return response.ToHttpResponse();
        }
    }
}

Administration

AdministrationController class code:

using System.Threading.Tasks;
using Microsoft.AspNetCore.Mvc;
using Microsoft.Extensions.Logging;
using ServiceMonitor.Core.BusinessLayer.Contracts;
using ServiceMonitor.WebAPI.Responses;

namespace ServiceMonitor.WebAPI.Controllers
{
#pragma warning disable CS1591
    [Route("api/v1/[controller]")]
    [ApiController]
    public class DashboardController : ControllerBase
    {
        protected readonly ILogger Logger;
        protected readonly IDashboardService Service;

        public DashboardController(ILogger<DashboardController> logger, IDashboardService service)
        {
            Logger = logger;
            Service = service;
        }
#pragma warning restore CS1591

        /// <summary>
        /// Gets service watcher items (registered services to watch with service monitor)
        /// </summary>
        /// <returns>A sequence of services to watch</returns>
        /// <response code="200"></response>
        /// <response code="500"></response>
        [HttpGet("ServiceWatcherItem")]
        [ProducesResponseType(200)]
        [ProducesResponseType(500)]
        public async Task<IActionResult> GetServiceWatcherItemsAsync()
        {
            Logger?.LogDebug("'{0}' has been invoked", nameof(GetServiceWatcherItemsAsync));

            var response = await Service.GetActiveServiceWatcherItemsAsync();

            return response.ToHttpResponse();
        }
    }
}

ServiceMonitor

This project contains all objects for Service Monitor Client, in this project, we have added the package Newtonsoft.Json for JSON serialization, in ServiceMonitor.Common there is an interface with name ISerializer and that's because I don't want to force to use a specific serializer, you can change that at this level. :)

ServiceMonitorSerializer class code:

using Newtonsoft.Json;
using ServiceMonitor.Common.Contracts;

namespace ServiceMonitor
{
    public class ServiceMonitorSerializer : ISerializer
    {
        public string Serialize<T>(T obj)
            => JsonConvert.SerializeObject(obj);

        public T Deserialze<T>(string source)
            => JsonConvert.DeserializeObject<T>(source);
    }
}

Next, we'll be working on MonitorController class, in this class, we'll perform all watching operations and save all results in database throught AdministrationController in Service Monitor API.

MonitorController class code:

using System;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.Extensions.Logging;
using ServiceMonitor.Clients;
using ServiceMonitor.Clients.Models;
using ServiceMonitor.Common;
using ServiceMonitor.Common.Contracts;

namespace ServiceMonitor
{
    public class MonitorController
    {
        public MonitorController(ILogger logger, IWatcher watcher, IServiceMonitorWebAPIClient client, AppSettings appSettings)
        {
            Logger = logger;
            Watcher = watcher;
            Client = client;
            AppSettings = appSettings;
        }

        public ILogger Logger { get; }

        public IWatcher Watcher { get; }

        public IServiceMonitorWebAPIClient Client { get; }

        public AppSettings AppSettings { get; }

        public async Task ProcessAsync(ServiceWatchItem item)
        {
            while (true)
            {
                try
                {
                    Logger?.LogTrace("{0} - Watching '{1}' for '{2}' environment", DateTime.Now, item.ServiceName, item.Environment);

                    var watchResponse = await Watcher.WatchAsync(new WatcherParameter(item.ToDictionary()));

                    if (watchResponse.Success)
                        Logger?.LogInformation(" Success watch for '{0}' in '{1}' environment", item.ServiceName, item.Environment);
                    else
                        Logger?.LogError(" Failed watch for '{0}' in '{1}' environment", item.ServiceName, item.Environment);

                    var serviceStatusLog = new ServiceStatusLogRequest
                    {
                        ServiceID = item.ServiceID,
                        ServiceEnvironmentID = item.ServiceEnvironmentID,
                        Target = item.ServiceName,
                        ActionName = Watcher.ActionName,
                        Success = watchResponse.Success,
                        Message = watchResponse.Message,
                        StackTrace = watchResponse.StackTrace
                    };

                    try
                    {
                        await Client.PostServiceEnvironmentStatusLog(serviceStatusLog);
                    }
                    catch (Exception ex)
                    {
                        Logger?.LogCritical(" Error on saving watch response ({0}): '{1}'", item.ServiceName, ex.Message);
                    }
                }
                catch (Exception ex)
                {
                    Logger?.LogCritical(" Error watching service: '{0}': '{1}'", item.ServiceName, ex.Message);
                }

                Thread.Sleep(item.Interval ?? AppSettings.DelayTime);
            }
        }
    }
}

Before running console application, make sure about these aspects:

1. ServiceMonitor database is available
2. ServiceMonitor database has the information for service categories, services, service watchers and users
3. ServiceMonitor API is available

We can check the returned value for url api/v1/Dashboard/ServiceWatcherItems:

{  
  "message":null,
  "didError":false,
  "errorMessage":null,
  "model":[  
    {  
      "serviceID":1,
      "serviceEnvironmentID":1,
      "environment":"Development",
      "serviceName":"Northwind Database",
      "interval":15000,
      "url":null,
      "address":null,
      "connectionString":"server=(local);database=Northwind;user id=johnd;password=SqlServer2017$",
      "typeName":"ServiceMonitor.Common.DatabaseWatcher, ServiceMonitor.Common, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null"
    },
    {  
      "serviceID":2,
      "serviceEnvironmentID":3,
      "environment":"Development",
      "serviceName":"DNS",
      "interval":3000,
      "url":null,
      "address":"192.168.1.1",
      "connectionString":null,
      "typeName":"ServiceMonitor.Common.PingWatcher, ServiceMonitor.Common, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null"
    },
    {  
      "serviceID":3,
      "serviceEnvironmentID":4,
      "environment":"Development",
      "serviceName":"Sample API",
      "interval":5000,
      "url":"http://localhost:5612/api/values",
      "address":null,
      "connectionString":null,
      "typeName":"ServiceMonitor.Common.HttpWebRequestWatcher, ServiceMonitor.Common, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null"
    }
  ]
}

As we can see, API returns all services for DefaultUser, please remember the concept about one user can subscribe more than one service to watch, obviously in this sample, our default user is suscribed to all services but we can change this link in ServiceUser table.

Program class code:

using System;
using System.Threading.Tasks;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.Logging;
using ServiceMonitor.Clients;
using ServiceMonitor.Clients.Models;
using ServiceMonitor.Common;
using ServiceMonitor.Common.Contracts;

namespace ServiceMonitor
{
    class Program
    {
        private static ILogger Logger;
        private static readonly AppSettings AppSettings;

        static Program()
        {
            Logger = LoggingHelper.GetLogger<Program>();

            var builder = new ConfigurationBuilder().AddJsonFile("appsettings.json");

            var configuration = builder.Build();

            AppSettings = new AppSettings();

            configuration.GetSection("appSettings").Bind(AppSettings);
        }

        static void Main(string[] args)
        {
            StartAsync(args).GetAwaiter().GetResult();

            Console.ReadLine();
        }

        static async Task StartAsync(string[] args)
        {
            Logger.LogDebug("Starting service monitor...");

            var client = new ServiceMonitorWebAPIClient();

            var serviceWatcherItemsResponse = default(ServiceWatchResponse);

            try
            {
                serviceWatcherItemsResponse = await client.GetServiceWatcherItemsAsync();
            }
            catch (Exception ex)
            {
                Logger.LogError("Error on retrieve watch items: {0}", ex);
                return;
            }

            foreach (var item in serviceWatcherItemsResponse.Model)
            {
                var watcherType = Type.GetType(item.TypeName, true);

                var watcherInstance = Activator.CreateInstance(watcherType) as IWatcher;

                await Task.Factory.StartNew(async () =>
                {
                    var controller = new MonitorController(Logger, watcherInstance, client, AppSettings);

                    await controller.ProcessAsync(item);
                });
            }
        }
    }
}

Once we have checked the previous aspects, now we proceed to turn console application, console output is this:

dbug: ServiceMonitor.Program[0]
      Starting application
sr trce: ServiceMonitor.Program[0]
      06/20/2017 23:09:30 - Watching 'Sample API' for 'Development' environment
trce: ServiceMonitor.Program[0]
      06/20/2017 23:09:30 - Watching 'Northwind Database' for 'Development' environment
trce: ServiceMonitor.Program[0]
      06/20/2017 23:09:30 - Watching 'DNS' for 'Development' environment
trce: ServiceMonitor.Program[0]
      06/20/2017 23:09:35 - Watching 'DNS' for 'Development' environment
trce: ServiceMonitor.Program[0]
      06/20/2017 23:09:37 - Watching 'Sample API' for 'Development' environment
trce: ServiceMonitor.Program[0]
      06/20/2017 23:09:39 - Watching 'DNS' for 'Development' environment
trce: ServiceMonitor.Program[0]
      06/20/2017 23:09:42 - Watching 'DNS' for 'Development' environment
trce: ServiceMonitor.Program[0]
      06/20/2017 23:09:43 - Watching 'Sample API' for 'Development' environment
trce: ServiceMonitor.Program[0]
      06/20/2017 23:09:45 - Watching 'DNS' for 'Development' environment
trce: ServiceMonitor.Program[0]
      06/20/2017 23:09:47 - Watching 'Northwind Database' for 'Development' environment
trce: ServiceMonitor.Program[0]
      06/20/2017 23:09:48 - Watching 'Sample API' for 'Development' environment
trce: ServiceMonitor.Program[0]
      06/20/2017 23:09:48 - Watching 'DNS' for 'Development' environment
trce: ServiceMonitor.Program[0]
      06/20/2017 23:09:51 - Watching 'DNS' for 'Development' environment
trce: ServiceMonitor.Program[0]
      06/20/2017 23:09:53 - Watching 'Sample API' for 'Development' environment
trce: ServiceMonitor.Program[0]
      06/20/2017 23:09:54 - Watching 'DNS' for 'Development' environment
trce: ServiceMonitor.Program[0]
      06/20/2017 23:09:57 - Watching 'DNS' for 'Development' environment

Now we proceed to check the saved data in database, please check ServiceEnvironmentStatus table, you'll get a result like this:

ServiceEnvironmentStatusID ServiceEnvironmentID Success WatchCount  LastWatch
-------------------------- -------------------- ------- ----------- -----------------------
1                          4                    1       212         2018-11-22 23:11:34.113
2                          1                    1       78          2018-11-22 23:11:33.370
3                          3                    1       366         2018-11-22 23:11:34.620

(3 row(s) affected)

How it works all together? Console application takes all services to watch from API and then starts one task per watch item in an infinite way inside of MonitorController, there is a delay time for each task, that interval is set in Service definition, but if there isn't a defined value for Interval, the interval is taken from AppSettings; so after of perform of Watch action, the result is saved on database through API and the process repeats itself. If you want to perform a watch operation for other types, you can create your own Watcher class.

Points of Interest

• DatabaseWatcher works with SQL Server, so how you connect to MySQL, PostgreSQL, Oracle and others DBMS? Create your Watcher class for specific DBMS and implements the interface IWatcher and write the code to connect for target database.
• Can we host service monitor in non Windows platforms? Yes, since .NET Core is cross platform We can host this project on Windows, Mac OS and Linux.
• As I know, there isn't native support for ASMX in .NET Core but we can monitor both kind of services, simply adding the rows in Service table, the ASMX ends with .asmx.
• Why console client and API aren't one single project? To prevent common issues on publishing, I think it's better to have two different projects because in that case We can run service monitor in one server and host API in another server.
• In this initial version, there isn't any configuration about security because it's better you add that implementation according to your scenario; you can make work this project with Windows Authentication, custom Authentication or add external service for authentication, that makes sense?

Code Improvements

• Add Identity Server
• Add notifications for administrators on critical errors during services watching (email, sms, etc.)
• I think it's better to have TypeName in ServiceCategory instead of ServiceWatcher
• Add UI project to show in a pretty way the status of services for end users, using some front-end frameworks such as Angular

You can check Projects section to know about on way improvements.

History

• 17^th January, 2017: Initial version
• 20^th June, 2017: Addition of environment for services
• 19^th October, 2017: Update for Services (Business Layer)
• 15^th July, 2018: Update for .NET Core 2
• 22^th November, 2018: Removing repository pattern
• 10^th January, 2019: Addition of Help Page for Web API
• 27^th January, 2019: Code refactor