The Goal
My personal goal with what I present in this article was to achieve the ability to self-host multiple HTTPS websites that, while in the prototype stage, are still usable by others, thus I want an Internet presence for these sites, but without having to pay for hosting and certificates.
Eliminating Cost
This was the first issue I wanted to confront. I'm currently paying $60/mo for a couple minimum tier Amazon EC2 instances, and, while a single SSL certificate for one year from places like SSL.com, GeoTrust, GoDaddy, and Comodo range from $17 to $69 per year, when I've got 5 or so sites I want to self-host, that adds up pretty quickly. Keep in mind I'm already paying $130/mo for broadband access through my cable provider. So, the two issues involving eliminating cost are:
- not paying for hosting
- not paying for SSL certificates
Caveats
Before hosting your own server, review the contract with your provider. For example, Verizon FiOS specifically forbid hosting servers unless you get their business service. While my cable provider has business plans, I don't see anything in my contract that prevents me from hosting my own web sites.
Another problem is, do you have a static public IP? If not, then hosting your own websites is just not going to happen unless you use a service like noip.com or duckdns.org, both use subdomains so your URL would be [yourname].no-ip.org or [yourname].duckdns.org, neither of which is desirable. Removing the no-ip.org, you guessed it, costs money!
Better Hardware
Unless you go for dedicated hardware (which at one point I was paying $100/mo for, and still even then the equipment was pretty low end), hosting is done on VMs and the minimum tier at, for example, Amazon, is pretty minimum - 30GB disk space, 1GB RAM. This gets eaten up pretty quickly by the OS itself, SQL Server, and email server, etc. I've got several laptops and desktops lying around that have much better specs and are, well, just sitting around.
Caveats
A hosting provider should provide backups of your VM, automatically handle physical equipment failures, and other benefits. Given that these are prototype websites, this is not at the top of my priority list and can be handled by other means. Of course, the fact that a truck came by and somehow snagged the cable running from the pole across the street and completely ripped off the cable, resulting in 5 days of downtime, well, that kind of outage is definitely a factor! And of course, there's having redundant equipment in case of a hardware failure, power outages to deal with, cats chewing on cables, and other issues that a host provider handles (usually) for you.
Simplicity and Control
I like to be in control (which is why I don't like flying) and I like the ease of updating a site when the site is running locally, rather than having to go through a publishing utility, FTP, or other remote deployment mechanism. As silly as it sounds, "copy and paste" of the web app (and I have automated tools to do that) just make life simpler. Yes, the one-click "Publish" feature in Visual Studio is pretty simple, but sometimes I just need to run a migration on the DB, make a simple tweak to a web page, or want to set up a beta test site without having to go through the whole process of creating another VM on the host provider.
Caveats
Non-standard publishing processes, possibly more error prone, more local tooling.
Problem #1: Running Multiple HTTPS Websites
It's relatively easy to host one website, though it can be a bit tricky getting:
netsh http add sslcert ipport=0.0.0.0:443 certstorename=Root certhash=[hash]
appid={[some GUID]}
to work. The most annoying problem is when you copy and paste the hash, that you aren't pasting hidden characters, which is a common problem.
The problem is, using netsh, you can only associate one certificate to port 443. And since:
- the certificate is associated with the domain name;
- the certificate is validated (the handshake process) against traffic coming in on the port rather than being qualified by the domain name;
- getting a certificate that can be applied to multiple domain names is prohibitively expensive;
this is not a usable solution. Being a late adopter, I quickly discovered that, on Windows 7, hosting multiple HTTPS websites on IIS is also not possible.
The Solution: Server Name Indication and IIS 8 or Higher
The solution is to use Server Name Indication:
Server Name Indication (SNI) is an extension to the TLS computer networking protocol by which a client indicates which hostname it is attempting to connect to at the start of the handshaking process. This allows a server to present multiple certificates on the same IP address and TCP port number and hence allows multiple secure (HTTPS) websites (or any other Service over TLS) to be served by the same IP address without requiring all those sites to use the same certificate. It is the conceptual equivalent to HTTP/1.1 name-based virtual hosting, but for HTTPS. The desired hostname is not encrypted, so an eavesdropper can see which site is being requested.
SNI is supported by IIS version 8 and higher:
which meant finally biting the bullet and upgrading to Windows 10.
Problem #2: Free SSL Certificates
There are several free SSL certificate providers: letsencrypt, comodo, sslforfree, etc. These provide basic SSL certificates, but they typically expire every 90 days. LetsEncrypt uses the Automated Certificate Management Environment protocol:
...for automating interactions between certificate authorities and their users' web servers, allowing the automated deployment of public key infrastructure at very low cost. It was designed by the Internet Security Research Group (ISRG) for their Let's Encrypt service. The protocol, based on passing JSON-formatted messages over HTTPS, has been published as an Internet Draft by its own chartered IETF working group.
InstantSsl (Comodo) requires that you generate a CSR and install the certificate.
Conversely, using ACME requires setting up a handler for the "ACME challenge", which is a GET request sent over HTTP to your domain on a specific path /.well-known/acme-challenge which requires that you respond with the challenge token provided by a handshake process with LetsEncrypt.
Caveats
As F-ES Sitecore pointed out here on CP:
LetsEncrypt has issued certs to over 10,000 PayPal phishing sites and security experts say it is ruining HTTPS as it removes the trustworthiness of https leaving people more vulnerable to attacks.
A quick search came up with this security news blurb:
During the past year, Let's Encrypt has issued a total of 15,270 SSL certificates that contained the word "PayPal" in the domain name or the certificate identity. Of these, approximately 14,766 (96.7%) were issued for domains that hosted phishing sites, according to an analysis carried out on a small sample of 1,000 domains, by Vincent Lynch, encryption expert for The SSL Store.
The point being, you may feel a bit queasy about using a service like LetsEncrypt and indirectly supporting this kind of behavior by the less ethical members of our digital community.
The Solution, Step 1: acme.net
There are a lot of tools out there (including this CodeProject article) for using LetsEncrypt on *nix systems (sort of makes sense, right?) There are very few tools for integrating LetsEncrypt with IIS. I ended up finding this project on GitHub, but the problem was that the author was using xproj (which has "gone away") instead of csproj files. Looking at the forks of oocx's GitHub repo, I discovered frankhommers acme.net fork, which uses csproj files. Yay!
The Solution, Step 2: Handling the ACME Challenge
The console app acme.net does all the handshaking with LetsEncrypt to obtain a token and then (in manual mode) it waits for you to set up your server to respond with the validation request that LetsEncrypt sends to your domain. In automatic mode, it assumes that your server is set up to respond to any challenge request LetsEncrypt sends. I wanted to use the second mode, to automate the process as fully as possible.
What acme.net creates for you is the file containing the full token with which LetsEncrypt expects you to respond. Here's some example filenames (every time you ask for an SSL certificate, LetsEncrypt uses a totally different challenge token):
Just for giggles, let's look at the contents of the first file:
That's what LetsEncrypt expects you to respond with. When it makes the GET request, the URL is appended with the everything to the left of the ".", so the URL path would look like (for this particular ACME challenge):
/.well-known/acme-challenge/-1mU4FF8ssT3TyFrsPf1r0tZ0mzra1krIFz4sDbzTb8
The steps, therefore, to respond to the challenge automatically is:
- Look for the starting URL path of
/.well-known/acme-challenge. - Extract the partial token.
- Load the full token from the file of the same name.
- Send that as a text response to the GET request.
A simple server to do this is:
using System;
using System.IO;
using System.Net;
using System.Text;
using System.Threading.Tasks;
namespace acme
{
public class ServerExceptionEventArgs : EventArgs
{
public Exception Exception { get; set; }
}
public class AcmeChallengeServer
{
public event EventHandler<ServerExceptionEventArgs> ServerException;
protected bool running = true;
protected HttpListener listener;
public void Start(string localIP)
{
listener = new HttpListener();
listener.Prefixes.Add("http://" + localIP + "/");
listener.Start();
Task.Run(() => WaitForConnection(listener));
}
public void Stop()
{
running = false;
listener.Stop();
}
private void WaitForConnection(object objListener)
{
while (running)
{
HttpListenerContext context;
try
{
context = listener.GetContext();
}
catch (HttpListenerException)
{
break;
}
catch (Exception ex)
{
ServerException?.Invoke(this, new ServerExceptionEventArgs() { Exception = ex });
break;
}
if (context.Request.RawUrl.StartsWith("/.well-known/acme-challenge"))
{
string challengeFile = context.Request.RawUrl.RightOfRightmostOf('/');
if (File.Exists(challengeFile))
{
string data = File.ReadAllText(challengeFile);
context.Response.StatusCode = 200;
context.Response.ContentType = "text/text";
context.Response.ContentEncoding = Encoding.UTF8;
byte[] byteData = Encoding.ASCII.GetBytes(data);
context.Response.ContentLength64 = byteData.Length;
context.Response.OutputStream.Write(byteData, 0, byteData.Length);
}
}
context.Response.Close();
}
}
}
}
Why This Works
For several reasons:
- The challenge is sent to http://[yourdomain], so to receive the challenge, the domain will be pointed to your public IP.
- Granted, anyone can append some random partial token to
/.well-known/acme-challenge/, but if you don't have a matching challenge file created by acme.net with the full token, then you know someone is doing something nefarious. Black list them! - And so what? Even if they get the full token of the challenge response, it doesn't relate at all to your certificate, it merely authenticated your server as the one for which LetsEncrypt will generate an SSL certificate. Once authenticated, LetsEncrypt will send to acme.net the pfx file containing your certificate.
Other Tools
While I'm on the subject of acme.net, Rick Strahl discusses some other tools for handling the ACME challenge on his blog. Personally, I think the UI app I present here is by far the simplest and most flexible, but that's my not so humble opinion.
Problem #3: Automating the Process
I want to be able to register certificates in either IIS or, if I'm running a single website and don't want to use IIS, using netsh. In fact, you can still run one website with netsh port 443 binding, and other HTTPS websites in IIS! To put all the pieces together, we need a workflow that does this:
The Solution, Step 1: Calling acme.net
It was straight forward to launch acme.net as a process, except for this line that hides the cursor:
private void GetPfxPasswordFromUser()
{
System.Console.CursorVisible = false;
This throws an exception when launched in a windowless process, so the code needed a minor tweak. Thank goodness for open source!
There are a lot of parameters that need to be passed to acme.net, and a couple depend on whether you're registering the certificate manually or using IIS. This is handled by the front end with the following radio buttons:
"Neither" is an option if you just want the pfx file, but don't want to use IIS or netsh binding.
The comments describe the parameters I'm using.
private Process LaunchAcmeDotNet(string domainName, string certPassword)
{
string staging = rbStaging.Checked ? "-s https://acme-staging.api.letsencrypt.org " : "";
string manualOptions = "-c manual-http-01 -i manual";
string iisOptions = "-c manual-http-01 -i iis";
string options = rbIIS.Checked ? iisOptions : manualOptions;
Process p = Helpers.LaunchProcess(@"acme.net\acme.exe",
String.Format(staging + "-a -j -d {0} -p {1} {2}", domainName, certPassword, options),
stdout => this.Invoke(() => tbLog.AppendText(stdout + CRLF)),
stderr => this.Invoke(() => tbLog.AppendText(stderr + CRLF)));
return p;
}
For simplicity, and particularly because it's a bit of a cart before the horse problem when registering a certificate to IIS for a server that isn't running yet because the IIS server is only listening to port 80, I'm using the little mini-server described above for the ACME challenge in both netsh and IIS registration.
The Solution, Step 2: Testing - Production vs. Staging
LetsEncrypt doesn't like it if you hit their production server too often for a new certificate, so they have provided a staging server that you can use for testing. Use it! I found that I had to do a lot of complete workflow testing to get everything right. This is handled in the code above and is exposed to you in the UI, as shown.
The Solution, Step 3: netsh
The acme.net handles IIS binding for you, but to do netsh binding requires programmatically importing the certificate and making a call to netsh.
netsh: Programmatically Importing the Certificate
Easy, but if you look at all the code I commented out, you can see the various rabbit holes that I went down, such as importing the certificate into the "My" store, exporting it, doing a repair, re-importing it. Turns out all that isn't necessary, but at one point it seemed necessary as I was debugging the whole process. And getting these flags right was a process of trial and error!
static string ImportCert(StoreName storeName, string certFile,
string password, out string certHash)
{
X509Certificate2 certToImport = new X509Certificate2(certFile, password,
X509KeyStorageFlags.Exportable | X509KeyStorageFlags.MachineKeySet |
X509KeyStorageFlags.PersistKeySet);
X509Store store = new X509Store(storeName, StoreLocation.LocalMachine);
store.Open(OpenFlags.MaxAllowed);
store.Add(certToImport);
store.Close();
certHash = certToImport.Thumbprint;
return certToImport.SerialNumber;
}
netsh: Binding
Binding the certificate to port 443 is also straight forward (as long as you don't get the dreaded "Error 1312"!)
static void RemoveBinding(Action<string> log)
{
Process p = Helpers.LaunchProcess("netsh", "http delete sslcert ipport=0.0.0.0:443",
(stdout) => log(stdout),
(stderr) => log(stderr));
p.WaitForExit();
}
static void AddNewBinding(string certHash, Guid appId, Action<string> log)
{
Process p = Helpers.LaunchProcess("netsh",
"http add sslcert ipport=0.0.0.0:443 certstorename=Root certhash=" +
certHash + " appid={" + appId.ToString() + "}",
(stdout) => log(stdout),
(stderr) => log(stderr));
p.WaitForExit();
}
Again, note the code comments, particularly in the add binding.
Cleaning Up the Old Certificate(s)
Something I notice that doesn't happen when binding to IIS is that acme.net doesn't remove the old certificate. In my implementation, I clean up the certificates before getting a new certificate. CAUTION! If an error occurs, or your site has constant traffic, removing the old certificate before the new one is successfully installed can result in your user experiencing certificate errors!
public static void RemoveCert(StoreName storeName, string subjectName)
{
X509Store store = new X509Store(storeName, StoreLocation.LocalMachine);
store.Open(OpenFlags.ReadWrite | OpenFlags.IncludeArchived);
X509Certificate2Collection certCollection = store.Certificates.Find
(X509FindType.FindBySubjectName, subjectName, false);
foreach(var cert in certCollection)
{
store.Remove(certCollection[0]);
}
store.Close();
}
The Solution, Step 4: When Do My Certificates Expire?
(There are a couple of sites that I don't want publicly known that are related to some client work.)
It's also helpful to know when your certificates are going to expire. One useful thing to do in the future would be to set up a service that automatically renews the certificates that are, say, within 15 days of expiring, but that's beyond the scope of this article.
protected void GetCertificatesIn(StoreName storeName, List<CertificateExpiration> certs)
{
X509Store store = new X509Store(storeName, StoreLocation.LocalMachine);
store.Open(OpenFlags.OpenExistingOnly);
foreach (var cert in store.Certificates)
{
if (cert.Issuer.Contains("Let's Encrypt Authority"))
{
certs.Add(new CertificateExpiration()
{
Subject = cert.Subject.RightOf("CN="),
ExpirationDate = cert.NotAfter,
});
}
}
store.Close();
}
Some fancy formatting for the above textbox (it helps if you assign a monospace font like Consolas to the textbox):
var certExpDates = GetCertificateExpirationDates();
int maxSubjectLength = certExpDates.Max(c => c.Subject.Length);
tbExpirationDates.AppendText(
String.Join(
CRLF,
certExpDates.OrderBy(c => c.ExpirationDate).
Select(c => c.Subject.PadRight(maxSubjectLength) + " : " + c.ExpirationDate)));
Problem #4: I Dislike the Command Line
As you've probably guessed at this point, I'm UI guy, not a command line / script / powershell guy.
Solution
So the whole thing is packaged in a nice UI:
Besides showing you some useful information like your local and public IP addresses, it's very simple to enter your domain name and certificate password.
Conclusion
One thing to mention -- you have to run the application as an administrator! Otherwise, enjoy using LetsEncrypt and multiple website hosting with IIS!
Also, the executable includes the binaries for acme.net (modified as described earlier) so you don't need to download that app separately.
History
- 4th July, 2017: Initial version
