Running Everything in Docker

Posted by Nathan Osman on August 23, 2017

Docker is a great way to run web applications and services in a container. With Docker, each container is isolated from the host which encourages composability and greatly improves security. In this article, I will describe how I prepared a number of services to run in Docker on one of my servers.

Docker Compose

One of the key components that made the transition easy was Docker Compose. Rather than writing a bunch of shell scripts to launch each container with the correct flags, Docker Compose allows you to describe the containers and their properties with a single YAML file. For example, George’s blog (which is powered by WordPress) uses the following configuration:

george-db:
  image: mysql:5.7
  container_name: george-db
  volumes:
    - /data/george-db:/var/lib/mysql
george:
  image: wordpress:4.8
  container_name: george
  environment:
    - WORDPRESS_DB_HOST=george-db
    - WORDPRESS_DB_PASSWORD=[redacted]
  labels:
    - "caddy.addr=george:80"
    - "caddy.domains=georgethedev.xyz"
  volumes:
    - /data/george:/var/www/html/wp-content

I’ll explain what the labels do in the next section. Note that there is no link between the containers since they are accessible to each other by hostname.

I have chosen to store persistent data for each container within a subdirectory of /data on the host. This allows me to take a snapshot of all containers at any point in time by stopping them and archiving the /data directory.

Caddy

I have 20+ services in docker-compose.yml that expose a port providing HTTP access. Ideally, a single reverse-proxy could be used for all of the services and also provide SSL termination. It would also be nice if the reverse-proxy integrated with Let’s Encrypt for automatic TLS certificate renewal.

Well, such a tool exists in the form of Caddy.

Although Caddy is a standalone binary, it is also a Go package. This means that third-party apps can be written that embed Caddy — this is where caddy-docker comes into play.

caddy-docker is an app that I have written which automatically reconfigures Caddy and gracefully restarts it every time a Docker container is started or stopped. How does caddy-docker know the domain name and address for each virtual host in the configuration file? This is where the container labels come into play.

The best way to understand how this works is by walking through an example:

  1. A new container is started.

  2. caddy-docker obtains the labels for the newly-started container.

  3. The labels are used to create a virtual host configuration similar to the following:

     myservice.com {
     gzip
     proxy / myservice:8000 {
         header_upstream Host {host}
         header_upstream X-Forwarded-For {remote}
         header_upstream X-Forwarded-Host {host}
         header_upstream X-Forwarded-Proto {scheme}
         header_upstream X-Real-IP {remote}
         header_upstream Connection {>Connection}
         header_upstream Upgrade {>Upgrade}
     }
 }

  4. Caddy (embedded in caddy-docker) is gracefully restarted.

  5. If TLS certificates for the domain names are not available, they are automatically obtained from Let’s Encrypt.

A similar process takes place when a container is stopped or removed.

Services

Now that I have described the environment, I will take a look at how some of the containers are configured. We already examined WordPress in the first section. I’ll omit the caddy.domains labels from the examples below.

caddy-docker

Naturally, it makes sense to run caddy-docker itself in a container. This is done as follows:

caddy-docker:
  image: nathanosman/caddy-docker
  container_name: caddy-docker
  environment:
    - ACME_EMAIL=[redacted]
  ports:
    - "80:80"
    - "443:443"
  volumes:
    - /var/run/docker.sock:/var/run/docker.sock
    - /data/caddy-docker:/var/lib/caddy-docker

Note that the app needs access to the Docker daemon in order to monitor when containers are started and stopped.

Gogs

Gogs provides a self-hosted alternative to GitHub. It requires a database and I have chosen to use PostgreSQL:

gogs-db:
  image: postgres:9.6
  container_name: gogs-db
  volumes:
    - /data/gogs-db:/var/lib/postgresql/data

The Gogs container uses the gogs/gogs image. In order to allow Git clients to connect via SSH, port 22 on the host must be forwarded to the container:

gogs:
  image: gogs/gogs
  container_name: gogs
  labels:
    - "caddy.addr=gogs:3000"
  ports:
    - "8022:22"
  volumes:
    - /data/gogs:/data

Jenkins

Jenkins is fairly easy to run in Docker. I recommend using the jenkins/jenkins image for the container:

jenkins:
  image: jenkinsci/jenkins
  container_name: jenkins
  labels:
    - "caddy.addr=jenkins:8080"
  ports:
    - "50000:50000"
  volumes:
    - /data/jenkins:/var/jenkins_home

Port 50000 is used by the slaves to connect to the master.

MediaWiki

I am currently using the synctree/mediawiki image for deploying MediaWiki. This image first requires a MySQL database:

wiki-db:
  image: mysql:5.7
  container_name: wiki-db
  volumes:
    - /data/wiki-db:/var/lib/mysql

The wiki container itself uses three volumes - one for LocalSettings.php (generated by the installer), one for extensions, and one for the images that are uploaded:

wiki:
  image: synctree/mediawiki
  container_name: wiki
  environment:
    - MEDIAWIKI_DB_HOST=wiki-db:3306
    - MEDIAWIKI_DB_PASSWORD=[redacted]
  labels:
    - "caddy.addr=discernment:80"
  volumes:
    - /data/wiki/LocalSettings.php:/var/www/html/LocalSettings.php
    - /data/wiki/extensions:/var/www/html/extensions
    - /data/wiki/images:/var/www/html/images

Note that you will need to create an empty LocalSettings.php before starting the container.