If you have been following our blog, you'll know that sometimes we blog about events, sometimes about projects, sometimes about travels, and sometimes about highly technical and very specific stuff. I hope that by the title of the article you have easily figured out which one is the case this time.
The context
During the last year we have been migrating much of our infrastructure to Docker containers, as we have mentioned in other posts. In our particular setup, we run several small containers in each host, and have a reverse nginx proxy, running in Docker as well, to route client requests to the corresponding app.
Another component we've been introducing to our infrastructure are certificates powered by Let's Encrypt, a brand new CA that provides free SSL certificates to promote secure connections along the web. Let's encrypt certificates are valid for 90-days, and the service encourages you to automate the renewal process.
Options for automation
Let's encrypt provides several options for automating certificate installation in the form of plugins. However, none of the options available allowed us to fully automate the process without any disruption to any of the services, and without requiring changes to the Docker images we were using for our apps or for the nginx.
Running with Docker
One of the options for installing certificates provided by Let's Encrypt is a Docker image, but it involves some limitations as it is:
Docker is an amazingly simple and quick way to obtain a certificate. However, this mode of operation is unable to install certificates or configure your webserver, because our installer plugins cannot reach it from inside the Docker container.
In other words, you can use the Docker container to download the certificate, but the actual install process requires a manual step.
Install certificates in the nginx container
The solution we devised together with Juan for this was defining a volume /etc/letsencrypt in the nginx proxy container where the certificates are stored, and mounting that volume in the Let's Encrypt Docker container via the volumes-from switch, so the certificates are automatically reachable by the proxy.
The last line in the script sends a SIGHUP to the nginx container so it reloads its configuation as needed.
The folder PROXY_DIR/www/DOMAIN, one for each domain, is used to store the proof file generated by letsencrypt. We mount the entire PROXY_DIR/www/ folder in /var/www in the proxy container, so it can serve the proof files, and route the requests for it to the local directory instead of forwarding them to the application in the nginx configuration, once for each upstream $host defined:
As a side note, the notation you see in the snippet above is go templating language, as we use docker-gen to dynamically generate the nginx configuration.
Automate all the things
The letsencrypt.sh snippet is added to cron.monthly, so certificates are renewed on a monthly based. This can be set up once per every domain to be registered in letsencrypt in the server, as each renewal does not interfere with the other.
A word of caution though: Let's Encrypt currently imposes severe restrictions to the number of certificates per domain it issues per week, so if you are modifying and testing this setup several times, you may find yourself locked out the platform.