Using Drone and Docker for Continuous Integration

Jun 3 2014

It all started with a whine. "Isn't there something better than Jenkins?" I'd had it with Jenkins' hungry runtime, difficult setup, build slave system, and shoddy plugins. Matt Farina pointed me to Drone.

Drone is a Docker-based continuous integration and delivery platform written in Go. The Drone folks have a hosted version, or you can run it on your own system, which is what I did.

What It Does

Drone provides an environment for running tests, builds, and deploys. It falls along the spectrum somewhere between Jenkins and TravisCI. Here's roughly how it works:

  1. Through Drone's web interface, you configure Drone to fetch a project from a Git repository and "run it." What exactly it does when you run it is defined in the next step.
  2. Your application contains a .drone.yml file, which tells Drone how to setup and run your application. I'll come back to this in a bit, but this particular step is extremely flexible.
  3. Next time you do a git push to your repository, Drone check out the repo into a freshly minted Docker image (of your choice) and then "run it".
  4. Based on your configuration, the result of a successful run may be a notification ("Build succeeded!"), the creation of a release binary, or even a deployment.
  5. Finally, Drone trashes the Docker container and re-sets so that it will be in a pristine and known state for the next build.

Two Example Workflows

Here are the workflows behind two projects I have in Drone. The first is simple. The second is pretty complicated.

A Simple Go Build

This project builds a simple Go program and then runs its tests. The goal is simply to make sure that this project is always buildable. The .drone.yml file does the following:

  1. Begin with the go1.2 Drone Docker image
  2. Install gpm and gpm-git
  3. Install the program's dependencies.
  4. Run go test
  5. Notifications are sent to me via email.

That's it.

A More Complex Example

In the more complex case, we have a large Go server that we deploy. The workflow works sorta like this:

  1. Begin with a custom Docker image that has gpm, goose, and Python fabric installed, along with a few other tools.
  2. Use a postgres service
  3. With GPM, install the project's dependencies
  4. Using created and goose, create a database in postgres and install the schema
  5. Run go test on the source
  6. Run go build and create the binary executable
  7. Run our integration tests against the server (still working on this)
  8. Create a package from the built files and the necessary other files
  9. Kick off a Fabric script that sends this into our Packer server and then deploys to AWS

Some of these steps are fairly complicated, but our .drone.yml file is still about 50 lines.

The Good

  • Drone is very powerful, and the .drone.yml file is adequate for doing just about everything I've wanted.
  • Drone also supports services. Through this mechanism, it can spin up additional Docker images that play a supporting role to your main Drone run. In my case, I create a Postgres database service and run integration tests. In this way I can verify database deployments as well as application code.
  • Since it's built on Docker, you can create custom Docker images that mirror your production environment.
  • It comes with built-in support for some common deployment and build processes (including posting to S3, sending messages to HipChat, and even deploying to PaaS services like Heroku)
  • The security model is sound. While it might seem to be a pain to copy SSH keys from Drone to various other places, it is totally the right way to do things.

The Bad

  • Documentation is lackluster. I've been trying to contribute some to help out a bit, and others are doing the same. So I'm sure the situation will improve.
  • Some parts are definitely still a work in progress. In particular, the caching system is a little rough. But honestly, it feels more polished than Jenkins, and I've still found it easier to work with.

The Bottom Line

I've tried CI systems ranging from Bamboo to Jenkins, and I always feel like I'm either pushing against the limits of the tech or tiptoeing through a mine field of possible misconfigurations. I don't feel that way with Drone. The "disposable" Docker environment is an absolutely perfect way to test code. And the flexibility offered in the system makes it powerful.