This is the third in a series of several posts on how to do way more than you really need to with Let’s Encrypt, certbot, and a good server. I use all of these things regularly but I’ve never taken the time to take them apart, look at how they work, and spend hours in Google trying in vain to figure out how to put them back together. It was inspired by a disturbing trend of ISP privacy violations and the shocking regulatory capture of the US Federal Communications Commission.
This post sets up all the backend security logic (minus headers) to harden Nginx or Apache. I’ve tried to provide an explanation of each component and good values to use (or the means to create your own). If you don’t have OpenSSL, most of this is meaningless.
- The Series so Far
- Code
- Note
- Primary Security Reference
- Primary Config File
- Specify Allowed TLS Versions
- Generate a List of Good Ciphers
- Specify ECDHE Curve
- Generate Diffie-Hellman Group
- Use Server Cipher Preference
- OCSP Stapling
- SSL Session
- Primary Config File Redux
- Before You Go
- Legal Stuff
The Series so Far
- Overview
- First Steps
- Tuning with OpenSSL
- Useful Headers
- Generating and Testing a Cert
- Automating Renewals
Things that are still planned but probably not soon:
- Updating OpenSSL
- CSP Playground
- Vagrant Examples (don’t hold your breath)
Code
You can view the code related to this post under the post-03-openssl-tuning tag. If you’re curious, you can also check out my first draft.
Note
I’m testing out some new tooling. This will be wotw-highlighter’s shakedown run. Let me know what you think about the syntax highlighting! I’m pretty excited because (no whammies) it should work well in AMP and normally.
I wrote the majority of the Apache examples with httpd in mind, i.e. from a RHEL perspective. If you instead use apache2, most of the stuff should still work, albeit maybe just a little bit different.
Primary Security Reference
I’ll be using the Qualys suggested configuration to set this up. Most of this stuff is explained elsewhere on the internet. I wanted to grok the whole process, so I wrote it up.
If you’re reading this more than, say, a month or two from its publication date, I’d strongly urge you to follow the documentation links to find the most current algorithms, best practices, and so on. Even if my minutae is current, you should always check sources when security is involved.
Primary Config File
This creates a single file to hold the common config.
As I said before, I like /etc/<server>/common/, YMMV.
Nginx
$ sudo touch /etc/nginx/common/ssl.conf |
Apache
So this probably won’t work without some TLC. Apache differentiates global vs scoped config, and some of the things I mention only work in one or the other. The official docs state scope per directive and I’ve tried to match that. However, I’m going to pretend like it will work without issue and hope no one says anything.
$ sudo touch /etc/httpd/common/ssl.conf |
You’ll also need to ensure you’ve got the right modules installed and running. Depending on your server’s distro and the version of Apache you’re running, installing and enabling modules is done differently.
$ which a2enmod && echo "apache2" || echo "httpd" |
- If you’re running
httpd, enable them by editing/etc/httpd/conf.modules.d/00-base.conf(or another file there; you might have togrepthem out). - If you’re running
apache2, enable them viaa2enmod.
You’ll need these modules:
mod_rewritemod_sslmod_socache_shmcbfor any caching (sessions, stapling)
$ eval "$(which apachectl && echo apachectl || echo httpd) -M" | grep -E "rewrite|shmcb|ssl" |
You might actually have to install additional external packages depending on how you get Apache, e.g. mod_ssl on RHEL systems.
Specify Allowed TLS Versions
Qualys says v1.2 is the only secure version. v1.3 is only a draft, so including it might be odd. If you’re truly desperate, v1.1 isn’t too bad. Don’t forget that Qualys writes the benchmark, so if you ignore that advice, your rating will take a hit.
Nginx
1 | ssl_protocols TLSv1.2; |
Apache
1 | SSLProtocol -all +TLSv1.2 |
Generate a List of Good Ciphers
You might check the current list to make sure this is up-to-date. You can also shorten this list; I was curious how it was built.
/save/the/qualys/list/somewhere | |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 | ECDHE-ECDSA-AES128-GCM-SHA256 |
We can use grep to search with a pattern from a -file, composed of newline-separated -Fixed strings, where each pattern matches the entire line (-x). All we need is the available ciphers. openssl ciphers returns a colon-separated list, so we can pass it through translate before searching it.
$ openssl ciphers |
If you didn’t get any results, you should probably spend the few days necessary to move everything to a platform at least from the last decade. Or maybe openssl didn’t get set up correctly.
Unfortunately, this doesn’t retain the order Qualys uses, and I’ve yet to figure out a good way to maintain the original order. A standard bash approach would be to commpare the two lists, but comm expects the lists to be sorted.
$ comm -12 /save/the/qualys/list/somewhere <( openssl ciphers | tr ':' '\n' ) |
Brace expansion doesn’t reduce the size by much, so I gave up after a few hours on that tangent. However, the important thing is that you now know what ciphers are available for you to use that also have the Qualys stamp of approval.
Nginx
I don’t think specifying all these is a great idea, but I actually have no idea.
1 | ssl_ciphers ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-SHA:ECDHE-ECDSA-AES256-SHA:ECDHE-ECDSA-AES128-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-SHA:ECDHE-RSA-AES256-SHA:ECDHE-RSA-AES128-SHA256:ECDHE-RSA-AES256-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES256-SHA256; |
Apache
I don’t think specifying all these is a great idea, but I actually have no idea.
1 | SSLCipherSuite "ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-SHA:ECDHE-ECDSA-AES256-SHA:ECDHE-ECDSA-AES128-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-SHA:ECDHE-RSA-AES256-SHA:ECDHE-RSA-AES128-SHA256:ECDHE-RSA-AES256-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES256-SHA256" |
Specify ECDHE Curve
Qualys suggests using a specific elliptic curve for ECDHE. Elliptic Curve Cryptography, like other algorithms, has both secure and insecure methods. Also like other algorithms, there are a few common, widely used components. Qualys recommends secp256r1 or P-256. Coincidentally, that curve has most likely been backdoored by government agencies for years and has even been put down by government agencies more recently.
I mention that because there’s basically no way to protect yourself from actors with superior tech. No matter what you do, there’s always someone with more money and decades of classified algebra and combinatorics to keep you grounded. However, unless some of that knowledge has been put to code and leaked recently, most of the cryptography mentioned here will probably prevent wardriving script kiddies from messing with you or your users for a few years.
If you’d like to use something a bit more secure, check out SafeCurves. Like before, we’ll need to check what’s available to us with the installed version of OpenSSL.
$ openssl ecparam -list_curves |
Comparing my list to SafeCurves yields zero safe curves. That’s because OpenSSL has rolled most of those into v1.1, which isn’t in stable channels yet (and probably won’t ever make it to LTS channels).
$ openssl version |
That means we have two options:
- Update OpenSSL manually and hope nothing system-critical actually needed the older version.
- Pick the best from what we’ve got for now.
The first involves a ton of extra work (it’s all fairly straightforward, just super involved), so I’m going to cover that in a later post, once I’m finished with the current slate. That leaves us with making due with what we’ve got. Mozilla recommends prime256v1, secp384r1, and secp521r1 for modern compatibility (actually for any compatibility).
$ openssl ecparam -list_curves | grep -E "prime256v1|secp384r1|secp521r1" |
Nginx
1 | ssl_ecdh_curve secp521r1:secp384r1:prime256v1; |
Apache
1 | SSLOpenSSLConfCmd Curves secp521r1:secp384r1:prime256v1 |
Generate Diffie-Hellman Group
Vanilla OpenSSL is susceptible to Logjam (among other things), so you’ll want to create a new Diffie-Hellman group. Qualys mentions this as well; basically, don’t use defaults.
Nginx
To generate,
$ sudo mkdir -p /etc/nginx/tls |
To use,
1 | ssl_dhparam /etc/nginx/tls/dhparams.pem; |
Apache
To generate,
$ sudo mkdir -p /etc/httpd/tls |
To use,
1 | SSLOpenSSLConfCmd DHParameter "/etc/httpd/tls/dhparams.pem" |
Use Server Cipher Preference
Having done all of this work to actually set up cipher precedence and curves and lots of other things, it’s important to actually specify we’d prefer it if clients would use our configuration instead of theirs. Assuming all methods are the same (which is actually a horrible assumption), we have more control with our configuration.
Nginx
1 | ssl_prefer_server_ciphers on; |
Apache
1 | SSLHonorCipherOrder on |
OCSP Stapling
OCSP Stapling makes things a little bit simpler for Let’s Encrypt. To verify a cert’s veracity, clients historically had to send a request to the CA, negotiate that, and then, knowing the cert was valid, hit the intended address. OCSP stapling allows your server to periodically timestamp its validity through the magic of digital signatures.
Nginx
This config requires also setting ssl_trusted_certificate. This will be handled later, once we actually request a cert.
1 2 | ssl_stapling on; |
It’s fairly common to see a resolver defined as well, and equally common to see it defined as Google.
1 | resolver 8.8.8.8 8.8.4.4; |
I’m not a huge fan of this because it routes everything to Google. If left out, the resolver defaults to your DNS. Theoretically, my DNS already knows a user is checking me out. Google doesn’t need to be involved for the same reasons OCSP stapling was created.
Apache
Apache doesn’t make it easy. Here’s a Stack Exchange thread that seems to cover the important stuff. This is the first of many times the new Apache cache will pop up, and, every single time, it requires manual setup.
1 2 | SSLUseStapling On |
SSL Session
As a fullstack dev who spends most of his time performing brute force data analysis and putting out fires, I honestly don’t have a good baseline for what is or is not a safe config. Mozilla might have abandoned open communication to court the favor of big corporations, but they still occasionally support the little dev (just not the end user). The Mozilla TLS Config Generator is a fantastic tool to generate actually strong defaults. After reading several hundred posts about Nginx and Apache hardening for a couple of weekends, I’ve come to recognize the Mozilla standards pretty well. That was my long-winded way of saying this is section is total copypasta.
Due to a security issue, Mozilla doesn’t recommend using session tickets. More recently, at least one named vulnerability has popped up regarding session tickets, so use them at your own risk.
Nginx
1 2 3 | ssl_session_timeout 1d; |
Apache
Apache again makes this difficult. It does look like disabling tickets is easy, so they’ve got that going for them, which is nice.
1 2 | SSLSessionCache "shmcb:ssl_scache(512000)" |
Primary Config File Redux
Nginx
/etc/nginx/common/ssl.conf | |
1 2 3 4 5 6 7 8 9 10 11 12 | ssl_protocols TLSv1.2; |
Apache
For the nth time, I’d like to reiterate that I haven’t actually tested this config. I will. Eventually.
/etc/httpd/common/ssl.conf | |
1 2 3 4 5 6 7 8 9 10 11 12 | SSLStaplingCache "shmcb:/path/to/ssl_stapling(32768)" |
Before You Go
Let’s Encrypt is a fantastic service. If you like what they do, i.e. appreciate how accessible they’ve made secure web traffic, please donate. EFF’s certbot is what powers my site (and basically anything I work on these days); consider buying them a beer (it’s really just a donate link but you catch my drift).
Legal Stuff
I’m still pretty new to the whole CYA legal thing. I really like everything I’ve covered here, and I’ve done my best to respect individual legal policies. If I screwed something up, please send me an email ASAP so I can fix it.
- The Electronic Frontier Foundation and
certbotare covered by EFF’s generous copyright. As far as I know, it’s all under CC BY 3.0 US. I made a few minor tweaks to build the banner image but tried to respect the trademark. I don’t know who thecertbotlogo artist is but I really wish I did because it’s a fantastic piece of art. - Let’s Encrypt is trademarked. Its logo uses CC BY-NC 4.0. I made a few minor tweaks to build the banner image but tried to respect the trademark.
- I didn’t find anything definitive (other than EULAs) covering Nginx, which doesn’t mean it doesn’t exist. Assets were taken from its press page.
- Apache content was sourced from its press page. It provides a full trademark policy.