docker-machine and qnap container station

So earlier this week woot.com had a QNAP TS-453mini for an awesome price.  I've been in the market for a new NAS to replace my self built one from more than half a decade ago.  One of the selling points that caused me to purchase it was the container station.  The container station lets your run Docker containers directly on the NAS.  If you've talked to me in the past 3 years, I've probably told you how I've drank all of the container kool-aid.  So of course this alone is reason enough for me to look into it.

The NAS came today and I put some old drives in it right now as I'm waiting on my secondary drive order (to try and make sure I hit different production runs).  So I started tinkering with the container station when I remembered about docker machine.

Docker machine is used to control multiple docker engines.  Typically it is used to create a virtual machine, or provision one on a cloud provider.  However, there is also a poorly documented driver named "none".  The none driver is just straight docker api over a tcp socket.  The container station exposes this and provides the certificates for authentication.

To get the certificates, go to the Preferences page in the Container Station.  From there select the Docker Certificate tab.  This page has some instructions on how to install the certs, but that'll only work if you only plan on connecting to the NAS.  So instead just hit the download button.

Now that we have the certs we can create the machine in docker-machine.  Of course replacing %nas ip% with the IP address or hostname of the NAS and %name% with whatever you want to refer to it as in docker machine.  In the following examples I've named the machine nas.

docker-machine create --driver=none --url=tcp://%nas ip%:2376 %name%

If we try to use this as is right now we'll get the following error.

$ docker-machine ls --filter name=nas
NAME   ACTIVE   DRIVER   STATE     URL              SWARM   DOCKER    ERRORS
nas    -        none     Running   tcp://nas:2376           Unknown   Unable to query docker version: Get https://nas:2376/v1.15/version: x509: certificate signed by unknown authority

To fix this we need to add and configure the certs that we downloaded earlier into the docker machine config.  To do that we need to cd into the directory that holds the config.  Once there, we need to extract cert.zip into that directory.

$ cd ~/.docker/machine/machines/nas
$ unzip ~/Downloads/cert.zip
Archive:  /home/grim/Downloads/cert.zip
 extracting: ca.pem                  
 extracting: cert.pem                
 extracting: key.pem                 

Now we just need to modify the AuthOptions section to point to the correct files. It should look like the following:

"AuthOptions": {
    "CertDir": "/home/grim/.docker/machine/machines/nas/",
    "CaCertPath": "/home/grim/.docker/machine/machines/nas/ca.pem",
    "CaPrivateKeyPath": "/home/grim/.docker/machine/machines/nas/key.pem",
    "CaCertRemotePath": "",
    "ServerCertPath": "/home/grim/.docker/machine/machines/nas/cert.pem",
    "ServerKeyPath": "/home/grim/.docker/machine/machines/nas/key.pem",
    "ClientKeyPath": "/home/grim/.docker/machine/machines/nas/key.pem",
    "ServerCertRemotePath": "",
    "ServerKeyRemotePath": "",
    "ClientCertPath": "/home/grim/.docker/machine/machines/nas/cert.pem",
    "ServerCertSANs": [],
    "StorePath": "/home/grim/.docker/machine/machines/nas"
}

Now we can enable the host in docker machine and run docker ps on it:

$ eval $(docker-machine env nas)
$ docker ps
Error response from daemon: client is newer than server (client API version: 1.24, server API version: 1.23)

On no!! What happened?!?  Well docker is usually pretty strict about having the same version of the client talk to the same version of the server.  Luckily we can work around this.

$ export DOCKER_API_VERSION=1.23
$ eval $(docker-machine env nas)
$ docker ps
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES

And it works!  Now we can treat the nas as any old docker host and let docker-machine manage it for us.

Local^wBitbucket Pipelines

So a while ago Bitbucket started a Beta program from a new feature called Pipelines, or more appropriately, Bitbucket Pipelines.  Being interested in CI/CD I of course submitted an application right away.  I was accepted, but then found out it only had support for Git.

As you may or may not know, I'm not really a big fan of Git (that flamewar is for another time and place) and prefer Mercurial.  So much so that nearly all of the 300ish repositories I have access to on Bitbucket are Mercurial.  So I was dead in the water and couldn't do anything.

Fast forward a few months and Bitbucket added Mercurial support to Pipelines. SCORE!

I started adding Pipelines support to one of my more simple projects and unfortunately found it extremely tedious to have to push to Bitbucket every time to see if I fixed the build.  So, like any good Open Source developer, I started working on a solution.

Pipelines is built on top of Docker and uses a YAML file to describe how the build works.  I've been using Docker for very long time now (I gave a talk on it in August 2014 for anyone that's curious) so I'm very comfortable with it.  That said, all I really needed to do was take the YAML file and turn it into some docker run commands.

So after a few hours of work I had a working version of what I later named local-pipelines.

It sat that way for awhile until Sean Farley ran into the same issues I did with not being able to test until pushing.  He then proceeded to clean the VCS interaction code and added support for passing environment variables into the pipeline.  His work got me working on it again too so I finally cleaned up the documentation and got it uploaded to pypi.

You can find more information on the overview page or if you're using MacPorts you can find it there as well.