Click here to Skip to main content
13,150,622 members (45,691 online)
Click here to Skip to main content
Add your own
alternative version

Stats

7.5K views
15 bookmarked
Posted 4 Sep 2017

asp.net core: implement a load balancer

, 8 Sep 2017
Rate this:
Please Sign up or sign in to vote.
.net core showcase: learn basics implementing a toy tool

Introduction

I spent some time into learn .net core functionalities and I wanted to test on a real project, so I start a new one. I don’t know why but I decided to implement a software load balancer. There are many option in the market and lot of them are opensource. The project start only to give me the opportunity for experimenting the framework so to reinvent the wheel don’t scared me.

I thought to a load balancer because it is managed in most of implementation by request filter according with the “pipeline pattern”. The middleware of .net core (or owin too…) are very similar so it seems the right application to go in deep on this technology.

 

What the Asp.Net Core Load Balancer will do

The behaviour of a load balancer is quite simple so I avoid wasting time explaining what a balancer is. Anyway I’ll spend few words describing how I decide to implement it.

Requirements:

  • Be plug and play: no complex installation

  • Be standalone or integrated in web server (nginx,apache,iis)

  • Changing configuration will provide: a proxy server, a balancing server, both of them

  • Use as much as possible what asp.net core gives out of the box

  • Keeping in mind performances

Modules

The main idea is to define a set of “modules” that can be activated or not basing on configuration. It have to be possible to add new modules and let third parties to develop their one.

Filters

This module provide an easy way to filter request based on some rules. All request that match the filter will be dropped. Each url is tested over a set of rule. If the url match the rule the request will be dropped. Only one match determine the rule activations so, basically, all rules are "OR" conditions by default. Each rule can test a set of request parameters (url, agent, headers). Inside the single rule all condition must be true to activate the rule. This means we are working with something like this ( CONDITION A AND CONDITION B) OR (CONDITION C) and this will support most cases.

Caching

By using standard .NET Core caching module we can provide cache support for url, defining policy,etc. Caching has many options that are basically a wrap of original module, so you ca refer to here for more details.

Rewrite

This stage will allow static rewrite rule. This is often demanded to the applications but can be implemented here to simplify server part or to map virtual urls over many different applications. This is mostly a way to couple external url with internal one in case there isnt' a way to change balanced application. Balancer itself will balance the output of this transformation.

Balancing

This is the core module that define, for each url wath will be the destination. This step generates only the real path, replacing selected host. The host can be selected using one of the following algorithm:

  1. Number of request coming

  2. Number of request pending

  3. Quicker response

  4. Affiliation (Based on Cookie)

Proxy

After Balancing stage complete the computation of right url, proxy module will invoke the request repling the client.

 

.NET CORE IN ACTION

In this section I’ll show most important asp.net core feature that I have used into this application to get the result.

The Host

.Net core provide two built in server that give you the capability to run a web application (Kestrel, Http.sys). The good part is that any application can run and act as a web server, and this is very interesting to run local angular application, maybe based on electron framework. The bad part is that in the most scenarios this have to run behind a proxy server due their limitation. The first limitation I have in mind is that Kestrel doesn’t support host binding, but only listen on ports. So, if you want to have two different web site in the same port it is a problem. For the balancer is not a problem, because the main feature is to get the whole traffic and then route it to destination, but on the real world web server have to provide multiple web sites on same port, so you probably will need to use IIS or any other solution again. Another pain is about HTTPS: the configuration on Kestrel is not so easy and dynamic. So also in this case stay behind a web proxy i preferable.

Some Kestrel settings are based on applicationsettings.json (i.e. server.urls) but can be setted also programmatically, here a complex example:

  1  public static IWebHost BuildWebHost(string[] args) 
  2  {
  3      WebHost.CreateDefaultBuilder(args)
  4          .UseStartup<Startup>()
  5          .UseKestrel(options =>
  6          {
  7              // some settings
  8              options.Limits.MaxConcurrentConnections = 100;
  9              options.Limits.MaxConcurrentUpgradedConnections = 100;
 10              options.Limits.MaxRequestBodySize = 10 * 1024;
 11              options.Limits.MinRequestBodyDataRate =
 12                   new MinDataRate(bytesPerSecond: 100, gracePeriod: TimeSpan.FromSeconds(10));
 13              options.Limits.MinResponseDataRate =
 14                   new MinDataRate(bytesPerSecond: 100, gracePeriod: TimeSpan.FromSeconds(10));
 15              //listening on http
 16              options.Listen(IPAddress.Loopback, 5000);
 17              // listening on 5001, but using https
 18              options.Listen(IPAddress.Loopback, 5001, listenOptions =>
 19              {
 20                  listenOptions.UseHttps("testCert.pfx", "testPassword");
 21              });
 22          })
 23          .Build();        
 24  }

On my opinion, to manage such settings as hard coded values are not the best option because are mainly configuration issues. Anyway to mix values from configuration and some hardcoded can lead to some situation hard to understand and I suggest to manage as mach as possible all settings in a single place.

Middlewares and plugin system

Middleware are a very nice system and it is easy to implement your own. This is a sample:

  1  public class MyMiddleware
  2      {
  3          //store here the delegate to execute next step
  4          private readonly RequestDelegate _next;
  5  
  6          //get the next step on ctor
  7          public RequestCultureMiddleware(RequestDelegate next)
  8          {
  9              _next = next;
 10          }
 11  
 12          //do something and then invoke next step
 13          public Task Invoke(HttpContext context)
 14          {
 15              //do things here
 16  
 17              // Call the next delegate/middleware in the pipeline
 18              return this._next(context);
 19          }
 20      }

The part with “next” call is used to invoke next steps on the pipeline.

A good practices is to create an extension method to allow the registration on Startup simply invoking it:

  1  public static class MyMiddlewareExtensions
  2      {
  3          public static IApplicationBuilder UseMyMiddleware(this IApplicationBuilder builder,
  4                  MyParam optionalParams)
  5          {
  6              return builder.UseMiddleware<MyMiddleware>();
  7          }
  8      }

There aren’t any limitation or rule to implement it: you just have to write the code inside a method. .The way I don’t like is that there is a lot of freedom and there are lot of things left to convention and to the implementor.Of course, in normal usage, we need only to introduce middleware yet done and interact with their configuration( Think about MVC one, you just have to include, then write files to let it work). In this application, because middleware are the main part and we introduce lot of them, I preferred to give a scaffold to let thid parts to develop new plugin without know how all other modules works. This is done by implementing an abstract class that give to the implementor a way to define:

  • if the plugin is active or not for the current request

  • if the request have to be terminated or can flow to next steps

  • write the code that do thinks ( is. in balancer middleware define wich server use as destination)

  • write the configuration

 

The implementation of the module is abstract so user have to implement, other method has a default implementation and can be omitted (standard beaviour is: active basing on settings, never stops flow, register itself on startup).


Here the abstract class definition. Default implementation are omitted to keep things readable, but you can inspect full source code.

  1   public abstract class FilterMiddleware:IFilter
  2   {
  3       public virtual bool IsActive(HttpContext context)
  4       {
  5          //compute here the logi based on httpcontext to tell if this stage is active or not
  6       }
  7       
  8       public override async Task Invoke(HttpContext context)
  9       {
 10          var endRequest = false;
 11          if (this.IsActive(context))
 12          {
 13              object urlToProxy = null;
 14              // compute args here...
 15              await InvokeImpl(context /* provide computed args here*/);
 16              endRequest = this.Terminate(context);
 17          }
 18  
 19          if (!endRequest && NextStep != null)
 20          {
 21              await NextStep(context);
 22          }
 23      }
 24      
 25      public virtual bool Terminate(HttpContext httpContext)
 26      {
 27          //compute logic to tell to inoke method if request can be terminated
 28          return false;
 29      }
 30  
 31      //create an instance of filter and register it
 32      public virtual IApplicationBuilder Register(IApplicationBuilder app, IConfiguration con, IHostingEnvironment env, ILoggerFactory loggerFactory)
 33      {
 34          return app.Use(next => 
 35          {
 36              var instance = (IFilter)Activator.CreateInstance(this.GetType());
 37              return instance.Init(next).Invoke;
 38          });
 39      }
 40        // Implementation of filter (must be implemented into child class)
 41       public abstract Task InvokeImpl(HttpContext context,string host, VHostOptions vhost,IConfigurationSection settings);
 42  }

The list of active plugins are written into config so that to add new one, without changing the main application, you just need to create your dll with the module, include it in bin folder with all dependencies and add an entry to config files.

This is the snippet to register all middlewares, the configuration is the topic of next paragraph, so I show here only the registration part.

  1   //BalancerSettings.Current.Middlewares contains all middleware read from config
  2   foreach (var item in BalancerSettings.Current.Middlewares)
  3   {
  4     item.Value.Register(app, Configuration, env, loggerFactory);
  5   }

Configuration

Main topic about configuration is that have to be dynamic and each middleware have to be able to read its part easily. I wanted also to use as much as possible the out-of-the-box way. Fortunately asp.net core configuration supports natively

  • json deserialization binding section to objects,

  • getting single value by path (navigating the json tree)

  • merging multiple settings file

  • dynamically apply one config basing on environment

Loading main settings

Main settings are store in a conf file and is binded with a singleton element shared across all application parts.

Here the json code:

  1  {
  2    "BalancerSettings": {
  3      "Mappings": [
  4        {
  5          "Host": "localhost:52231",
  6          "SettingsName": "site1"
  7        }
  8      ],
  9      "Plugins": [
 10        {
 11          "Name": "Log",
 12          "Impl": "NetLoadBalancer.Code.Middleware.LogMiddleware"
 13        },
 14        {
 15          "Name": "Init",
 16          "Impl": "NetLoadBalancer.Code.Middleware.InitMiddleware"
 17        },
 18        {
 19          "Name": "RequestFilter",
 20          "Impl": "NetLoadBalancer.Code.Middleware.RequestFilterMiddleware"
 21        },
 22        {
 23          "Name": "Balancer",
 24          "Impl": "NetLoadBalancer.Code.Middleware.BalancerMiddleware"
 25        },
 26        {
 27          "Name": "Proxy",
 28          "Impl": "NetLoadBalancer.Code.Middleware.ProxyMiddleware"
 29        }
 30      ]     
 31      
 32    }

Here the code to bind it to the class, using dependency injection to make it available on all constructors.

  1   public void ConfigureServices(IServiceCollection services)
  2   {
  3     services.AddOptions();
  4     services.AddMemoryCache();
  5     services.Configure<BalancerSettings>(Configuration.GetSection("Balancersettings"));
  6   }
  7  
  8   public void Configure(IApplicationBuilder app, IHostingEnvironment env, ILoggerFactory  loggerFactory,
  9                          IOptions<BalancerSettings> init)
 10      //here you can handle injected value
 11   }

 

Apply dynamic config basing on requests

This feature covered most of the issues, but I still need a way to apply different configuration basing on request data. Yes, because all setting are static and we cannot run multiple instance of application with different settings. A solution would be to replicate the logic to get contextualized data into each middleware but this way didn’t like because it will ask us to replicate lot of logics in many classes. Basically, if I am serving site1.com I have to take different settings then siste2.com. Such rules usually is managed as application data, like storing in a database. But in this case, I wanted to use only configuration to introduce less components as possible.


The solution I found, uses all standard  feature and needs only config files. First of all i have a map that define for all host name the configuration file name. This allow to share same config across multiple domains, i.e. telling that www.site1.com and site1.com must route to same cluster.

  1   "Mappings": [
  2        {
  3          "Host": "<a href="http://www.site1.com/">www.site1.com</a>",
  4          "SettingsName": "mycluster"
  5        },
  6        {
  7          "Host": "site1.com",
  8          "SettingsName": "mycluster"
  9        }
 10      ]

All configuration are named and linked by reference from the previous schema.

During the request processing, I get the host  from request value so I can read the configuration section relate to it. This is the few methods that reads the section from host, resolving by configuration name.

  1  //get settings name from host (www.site.com=>mybalancer)
  2  public string GetSettingsName(string host)
  3  {
  4      //in memory map that reflects .json settings
  5      return hostToSettingsMap[host];
  6  }
  7  
  8  //get section from hostname (www.site.com=> read mybalancer section)
  9  public IConfigurationSection GetSettingsSection(string host)
 10  {
 11      string settingsName = GetSettingsName(host);
 12      return Startup.Configuration.GetSection(settingsName);
 13  
 14  }
 15  
 16  // bing settings to the class of a given type T
 17  public T GetSettings<T>(string host) where T : new()
 18  {
 19      var t = new T();
 20      GetSettingsSection(host).Bind(t);
 21      return t;
 22  }

So, all middleware has access to the configuration relate to the current host and can find inside it their own section. See the balancer that read its’ information as example:

  1  //Balancer implementation
  2  public async override Task InvokeImpl(HttpContext context, string host, VHostOptions vhost, IConfigurationSection settings)
  3  {
  4    BalancerOptions options= new BalancerOptions();
  5    settings.Bind("Settings:Balancer", options);
  6    //.. continue doing real work..
  7  }

This is the part when i put all config files together:

  1  public Startup(IConfiguration configuration,IHostingEnvironment env)
  2  {
  3      Configuration = configuration;
  4  
  5      var builder = new ConfigurationBuilder()
  6         .SetBasePath(env.ContentRootPath)
  7         .AddJsonFile("conf/appsettings.json", optional: true, reloadOnChange: true);
  8  
  9      //get all files in ./conf/ folder
 10      string[] files = Directory.GetFiles(Path.Combine(env.ContentRootPath, "conf", "vhosts"));
 11  
 12      foreach (var s in files)
 13      {
 14          builder = builder.AddJsonFile(s);
 15      }
 16  
 17      builder=builder.AddEnvironmentVariables();
 18      Configuration = builder.Build();
 19  }

Logging

Logging isn’t a new feature into programming and in .net framework there some tool to do out of the box. The good news is that, today, we have a very complete logging framework that work in the way we like  (NLog, Log4net). Logs can be routed to the default provider or to third parts framework like NLog, that I used into this project. The logger is provided from DI into constructor and as many things in .net core the best practices is to store into a local variable, something like this:

  1  public class MyController : Controller
  2  {
  3      private readonly ILogger _logger;
  4  
  5      public TodoController(ILogger<MyController> logger)
  6      {     
  7          _logger = logger;
  8      }
  9  }

 

To use an external provider is easy, here my config that send logs to Nlog.

  1  loggerFactory.AddNLog();
  2  app.AddNLogWeb();
  3  env.ConfigureNLog(".\\conf\\nlog.config"); //I decide to place here...
 


Point of Interest

 

Is .net core ready for production?

Lot of people told me asp.net core is not ready for the market because it is lot younger then regular framework. Of, course .Net framework is a very mature framework, improved on each release and give us a lot of certaines. It's also true that, comparing with .net core, it is a lot more mature. By the way, this doesn’t means .net core is  not enough to be used in production. If you remember on late 2001, when .net framework come out, it was not so mature too, but in 2005, just after couple of year from its born, .net 2.0 was very reliable and  have been chosen as the best solution in a large amount of project (I remember also version 1.1 that was working after a couple of hotfix and minor releases…).  Also for .net core the first year has gone and I found on it must of the feature I need. There are a lot of thirdy party library that are now available on .net core too and other are going to be ported. So, if you are looking for a technology to develop a long term project it is an option to take in account. Even more, if you are going to implement a multi platform one, it is a very good solution to bring .net power (c# and Visual Studio) on every workstation or servers.

When chose .net core

  • No dependency from .net assembly or third party library available only on .net

  • Need to implement a cross platform application

  • Starting an application that may need in future one of the above point

  • Implement a local server to create a client application based on angular\electron

  • implement a pure api \microservice application

  • Delploy on Docker

  • Want to experiment

 

When chose .net framework

  • Have any .net framework dependency (libraries or projects)

  • Have to use COM object or any platform dependent technology

 

Next steps

As this is a functionally working load balancer there are some further step to make it ready for the market. Of course there may be a long list of things to do but, excluding the feature development, we can summarize to:

  • marking performance tuning and load test

  • package it, releasing multiple bundle depending on  OS and mode.

History

  1. 2017-09-05: First version published 

License

This article, along with any associated source code and files, is licensed under The GNU General Public License (GPLv3)

Share

About the Author

Daniele Fontani
Technical Lead
Italy Italy
I'm senior developer and architect specialized on portals, intranets, and others business applications. Particularly interested in Agile developing and open source projects, I worked on some of this as project manager and developer.

My programming experience include:

Frameworks \Technlogies: .NET Framework (C# & VB), ASP.NET, Java, php
Client languages:XML, HTML, CSS, JavaScript, angular.js, jQuery
Platforms:Sharepoint,Liferay, Drupal
Databases: MSSQL, ORACLE, MYSQL, Postgres

You may also be interested in...

Comments and Discussions

 
BugIt's KESTREL Pin
rendle6-Sep-17 0:00
memberrendle6-Sep-17 0:00 
GeneralRe: It's KESTREL Pin
Daniele Fontani6-Sep-17 1:54
professionalDaniele Fontani6-Sep-17 1:54 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

Permalink | Advertise | Privacy | Terms of Use | Mobile
Web04 | 2.8.170924.2 | Last Updated 8 Sep 2017
Article Copyright 2017 by Daniele Fontani
Everything else Copyright © CodeProject, 1999-2017
Layout: fixed | fluid