This is the first in a series of several posts on how to do way more than you really need to with Let’s Encrypt, certbot, and a good server. I use all of these things regularly but I’ve never taken the time to take them apart, look at how they work, and spend hours in Google trying in vain to figure out how to put them back together. It was inspired by a disturbing trend of ISP privacy violations and the shocking regulatory capture of the US Federal Communications Commission.

This post begins with the sad state of affairs that is the current US internet landscape but quickly moves on to more interesting topics like background information on HTTP, HTTPS, HSTS, Let’s Encrypt, and certbot. It’s intended as a gentle introduction and mainly served as a way for me to define what it was I was trying to accomplish.

The Series so Far

  1. Overview
  2. First Steps
  3. Tuning with OpenSSL
  4. Useful Headers
  5. Generating and Testing a Cert
  6. Automating Renewals

(This section should get updated as series progresses.)

Code

You can view the code related to this post under the post-01-overview tag.

The Status Quo

2017’s been a really scary year for the internet. Corporations strong-armed the W3C into adding black box DRM to the HTML spec. WPA2 was cracked. IoT botnets, while not making much of a splash this year, are now a regular threat. The US Federal Communications Commissions seems to be in the final stages of regulatory capture.

The fight for US net neutrality has been center stage, at least everywhere I look, for the past month. So much so that I almost missed this very interesting story about Comcast snooping unencrypted traffic. Within a day or two of reading the original post, I stumbled on this thread illustrating the same problem within Steam (apparently Steam never learned). Apparently ISPs have been injecting code for years. That’s not okay.

Sidestepping some of these issues takes a small investment of time up front, but is quickly automated afterward. The pernicious attitude that HTTPS isn’t really necessary needs to go away, and this is a great place to start.

Why Introduce This?

Before looking at how to build a strong configuration, it’s worth some time to investigate the components involved. It’s hard to understand why HTTPS is important without understanding what distinguishes it from HTTP. Without some historical perspective, Let’s Encrypt seems like just the right thing and natural thing to do (it is) instead of the daring breakthrough it was and amazing disruptive business model it’s become.

HTTP vs HTTPS

The primary difference between HTTP and HTTPS is encryption. The former is transmitted in the clear; the latter is encrypted prior to transmission. As web traffic flows through many nodes between source and destination, there are many opportunities for tampering or sniffing. HTTP neither has the ability to prevent attacks like this nor the hindsight to know they occurred. HTTPS defeats tampering and sniffing via symmetric-key cryptography (assuming, of course, the attacker does not have access to sufficiently superior hardware).

However, HTTPS isn’t just for people that need to pass secrets. It adds an extra layer of authenticity, giving your users some confidence they’re actually communicating with you. To an extent, it keeps communication and activity private. HTTPS means the parties involved, and (theoretically) only the parties involved, will communicate.

Serving HTTP content is as simple as throwing something on a public address (well, with DNS and all that too, but I’m doing simple here). Serving HTTPS content requires more tooling. The box in question needs a digital identity (a cert) that will be used to establish secure pipelines. While you can technically issue one yourself, the internet usually expects a third party to be involved (and by “usually” I mean “self-signed certs are never accepted”). After obtaining digital ID, the content has to be served via encryption libraries (e.g. the indomitable OpenSSL) and consumed by user agents capable of handling the encrypted tunnels (glossing over some refactoring that inevitably must be done to fix protocol-aware content). Modern webservers and browsers make the entire exchange fairly straightforward.

To make life easier, HTTPS content is usually served with additional HTTP pointers to the secure content, which cover user agents that don’t try HTTPS by default. Nine times out of ten that means http://example.com/page gets a 301 Moved Permanently that points to https://example.com/page (and I’m not sure what happens the other one time). HTTP and HTTPS are two very different protocols (rather, application layer v.s. transport layer), so you can’t serve HTTPS as HTTP. Instead, you instruct the user to resend the request using HTTPS.

HSTS

HTTP Strict Transport Security (HSTS) is a web standard that instructs user agents to use strict HTTPS. Its support is pretty universal. The HSTS header automatically redirects HTTP traffic to HTTPS, adding another layer of enforcement. If it encounters an invalid HTTPS configuration (e.g. cert errors), HSTS prevents users from accessing the page entirely (e.g. this intentional error page). It’s cached by the browser, not the server, so an attacker can’t just remove it from your content and redirect to a spoofed site.

HSTS can make sites a bit more complicated. It’s recommended to cover subdomains, but that can be complicated on larger sites. Wildcard certs and HSTS can actually track everything ever, so you have to be aware of what you’re loading externally. Finally, attackers aren’t the only ones that can break HSTS. If your configuration breaks (e.g. forgot to renew the cert), users are locked out until you fix it.

At its core, HSTS is intended to snag some low-hanging fruit. By enforcing a site-wide HTTPS policy, shady third-party code can’t hijack content (sort of). It’s also much harder to execute a man-in-the-middle attack of common varieties against a site with HSTS declared. More importantly, it forces users to switch to secure traffic (which probably could have been done years ago if certs weren’t such a racket), all without making their eyes glaze over trying to figure out how to be safe. It just works.

Let’s Encrypt

From its homepage,

Let’s Encrypt is a free, automated, and open Certificate Authority.

No one should have to pay for secure communication. I don’t want to say much more than that, because I’ve got some pretty strong opinions about predatory shared hosting providers and the pervasive desire to pull the wool over everyone’s eyes that is central to the commercial SSL market.

Let’s Encrypt provides a free alternative. As a solo dev (at least with my Wizards consulting), it’s hard to describe just how much that changed my life. I can give local clients real security without even worrying about the cost (I mean, I do send them a few emails detailing both how much they’re saving and how they can donate; pay it forward). I can throw up a prototype for the few dollars a month a cheap VPS costs and actually provide a secure service without diverting student loan payments.

certbot

The Electronic Frontier Foundation has spearheaded an amazing tool to set up and deploy Let’s Encrypt certs anywhere (technically POSIX only but also technically you can make it work with a virtual machine and some elbow grease). They took an awesome idea and made it even more awesome. How neat is that?

Before You Go

Let’s Encrypt is a fantastic service. If you like what they do, i.e. appreciate how accessible they’ve made secure web traffic, please donate. EFF’s certbot is what powers my site (and basically anything I work on these days); consider buying them a beer (it’s really just a donate link but you catch my drift).

I’m still pretty new to the whole CYA legal thing. I really like everything I’ve covered here, and I’ve done my best to respect individual legal policies. If I screwed something up, please send me an email ASAP so I can fix it.

  • The Electronic Frontier Foundation and certbot are covered by EFF’s generous copyright. As far as I know, it’s all under CC BY 3.0 US. I made a few minor tweaks to build the banner image but tried to respect the trademark. I don’t know who the certbot logo artist is but I really wish I did because it’s a fantastic piece of art.
  • Let’s Encrypt is trademarked. Its logo uses CC BY-NC 4.0. I made a few minor tweaks to build the banner image but tried to respect the trademark.
  • I didn’t find anything definitive (other than EULAs) covering Nginx, which doesn’t mean it doesn’t exist. Assets were taken from its press page.
  • Apache content was sourced from its press page. It provides a full trademark policy.