Docker: A Health-Check Story!

Background

We use Docker to create and manage software packages and it carries two concepts: images and containers. We define and build portable Docker images to be shipped as software solutions. Docker containers are created as writable executable layers from those images, which are then deployed using cloud service providers.

Sometimes we need to ensure our containers are in a healthy status and are up and running when we deploy our application services, so we need to run some health checks. Here, we explain the existing methods to do that and we also review some hints & tips!

What is a Docker health-check anyway?

It is a way to determine if a running container is in a healthy status or not! Basically, we ask Docker to test if our application/service is up and running, by providing a series of commands for Docker to run. Docker runs those commands and checks the returned status code and then decides if the container is healthy or not.

In order to health-check a container, we can either add a HEALTHCHECK instruction to the Dockerfile, or we can add a healthcheck configuration parameter to the docker-compose.yml file if we use Docker Compose to manage a multi-container application.

For the following examples, we choose nginx (Nginx web server) as our base image and we use curl to call http://localhost as a simple check to see if it is up and running.

With Dockerfile

The HEALTHCHECK Instruction tells Docker how to test a container’s health status by running a specified command (e.g. curl, wget, sh) within a specified time interval and considering a certain timeout. This is an example of how we can add a health check for nginx:

If nginx is working fine, the curl — fail http://localhost || exit 1 command will return exit code 0 upon a successful request which tells Docker the container is healthy. Otherwise it will return exit code 1 which tells Docker the container has a problem and is unhealthy at this stage.

Hint: In Unix-based systems, there is a certain parameter we can pass to echo which then returns exit status of the last executed command: $?

shell$ curl -I http://localhost
HTTP/1.1 200 OK
Server: nginx/1.19.6
...
shell$ echo $?
0

Now, if we curl a non-existent endpoint, we will get another exit code:

shell$ curl -I http://localhost:81
curl: (7) Failed to connect to localhost port 81: Connection refused
shell$ echo $?
7

We have also asked Docker to:

• perform this check after start-period seconds as bootstrap time for the container,
• re-execute the check, interval seconds after a previous check,
• times out after timeout seconds,
• and consider the container unhealthy after retries consecutive failures.

Now, we build our image and create a container:

shell$ docker build -t nginx-health-test .
shell$ docker run -dp 80:80 nginx-health-test:latest

Now if you run docker ps -a (or docker container ls) and check the status — i.e. STATUS — of the created container, you will see the health-check process has started:

CONTAINER ID   STATUS
3b8df3ad0ba3   Up 1 second (health: starting)

And after a few seconds, you will see the status changes to healthy if there is no problem with the container:

CONTAINER ID   STATUS
3b8df3ad0ba3   Up 35 seconds (healthy)

Hint: The output format of the containers list can be customized:

docker ps -a — format="table {{.ID}}\t{{.Names}}\t{{.Image}}\t{{.Command}}\t{{.Status}}”

The HEALTHCHECK instruction can also have another form — i.e. NONE — which actually asks Docker to ignore and disable any health check inherited from the base image:

With Docker Compose file

Docker Compose is used to manage multi-container Docker applications. In a compose file, we can define our application’s services and configure them according to our requirements.

For this example, consider we have a web application which requires nginx as its web server. So, we need to define 2 services: app and nginx.

A similar approach to what we did in the Dockerfile can be done to add a health check to the compose file. We can add a Compose configuration parameter to our nginx service to define the check process:

We can now run docker-compose up -d and check the list of our containers using docker ps -a:

And again after a few seconds, you will see the Up About a minute (healthy) message.

Custom Health Check

So what if we need a more complex and comprehensive health check? Does curl http://localhost suffice for all image containers and scenarios? The answer is no!

As you may have guessed, for services like Nginx or ElasticSearch, a simple HTTP request could be enough to check if that service container is healthy (up & running) or we need to fix some issues and create another container.

But if we need to test a few more aspects of an application’s health, we can write a custom health-check script and run it:

HEALTHCHECK --interval=10s CMD go run healthcheck.go

Another option is we can build a specific status-check endpoint in our backend system and call that endpoint to verify that our application is healthy:

HEALTHCHECK --interval=10s \ 
  CMD curl --fail -I http://localhost:8080/status || exit 1

The last point to remember is that Docker does not stop unhealthy containers automatically and it is your job to monitor those containers, resolve any issues and restart the containers.

Happy coding!