Friday, February 3, 2017

docker-machine and qnap container station

So earlier this week woot.com had a QNAP TS-453mini for an awesome price.  I've been in the market for a new NAS to replace my self built one from more than half a decade ago.  One of the selling points that caused me to purchase it was the container station.  The container station lets your run Docker containers directly on the NAS.  If you've talked to me in the past 3 years, I've probably told you how I've drank all of the container kool-aid.  So of course this alone is reason enough for me to look into it.

The NAS came today and I put some old drives in it right now as I'm waiting on my secondary drive order (to try and make sure I hit different production runs).  So I started tinkering with the container station when I remembered about docker machine.

Docker machine is used to control multiple docker engines.  Typically it is used to create a virtual machine, or provision one on a cloud provider.  However, there is also a poorly documented driver named "none".  The none driver is just straight docker api over a tcp socket.  The container station exposes this and provides the certificates for authentication.

To get the certificates, go to the Preferences page in the Container Station.  From there select the Docker Certificate tab.  This page has some instructions on how to install the certs, but that'll only work if you only plan on connecting to the NAS.  So instead just hit the download button.

Now that we have the certs we can create the machine in docker-machine.  Of course replacing %nas ip% with the IP address or hostname of the NAS and %name% with whatever you want to refer to it as in docker machine.  In the following examples I've named the machine nas.

docker-machine create --driver=none --url=tcp://%nas ip%:2376 %name%

If we try to use this as is right now we'll get the following error.

$ docker-machine ls --filter name=nas
NAME   ACTIVE   DRIVER   STATE     URL              SWARM   DOCKER    ERRORS
nas    -        none     Running   tcp://nas:2376           Unknown   Unable to query docker version: Get https://nas:2376/v1.15/version: x509: certificate signed by unknown authority

To fix this we need to add and configure the certs that we downloaded earlier into the docker machine config.  To do that we need to cd into the directory that holds the config.  Once there, we need to extract cert.zip into that directory.

$ cd ~/.docker/machine/machines/nas
$ unzip ~/Downloads/cert.zip
Archive:  /home/grim/Downloads/cert.zip
 extracting: ca.pem                  
 extracting: cert.pem                
 extracting: key.pem                 

Now we just need to modify the AuthOptions section to point to the correct files. It should look like the following:

"AuthOptions": {
    "CertDir": "/home/grim/.docker/machine/machines/nas/",
    "CaCertPath": "/home/grim/.docker/machine/machines/nas/ca.pem",
    "CaPrivateKeyPath": "/home/grim/.docker/machine/machines/nas/key.pem",
    "CaCertRemotePath": "",
    "ServerCertPath": "/home/grim/.docker/machine/machines/nas/cert.pem",
    "ServerKeyPath": "/home/grim/.docker/machine/machines/nas/key.pem",
    "ClientKeyPath": "/home/grim/.docker/machine/machines/nas/key.pem",
    "ServerCertRemotePath": "",
    "ServerKeyRemotePath": "",
    "ClientCertPath": "/home/grim/.docker/machine/machines/nas/cert.pem",
    "ServerCertSANs": [],
    "StorePath": "/home/grim/.docker/machine/machines/nas"
}

Now we can enable the host in docker machine and run docker ps on it:

$ eval $(docker-machine env nas)
$ docker ps
Error response from daemon: client is newer than server (client API version: 1.24, server API version: 1.23)

On no!! What happened?!?  Well docker is usually pretty strict about having the same version of the client talk to the same version of the server.  Luckily we can work around this.

$ export DOCKER_API_VERSION=1.23
$ eval $(docker-machine env nas)
$ docker ps
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES

And it works!  Now we can treat the nas as any old docker host and let docker-machine manage it for us.

Friday, August 26, 2016

Local^wBitbucket Pipelines

So a while ago Bitbucket started a Beta program from a new feature called Pipelines, or more appropriately, Bitbucket Pipelines.  Being interested in CI/CD I of course submitted an application right away.  I was accepted, but then found out it only had support for Git.

As you may or may not know, I'm not really a big fan of Git (that flamewar is for another time and place) and prefer Mercurial.  So much so that nearly all of the 300ish repositories I have access to on Bitbucket are Mercurial.  So I was dead in the water and couldn't do anything.

Fast forward a few months and Bitbucket added Mercurial support to Pipelines. SCORE!

I started adding Pipelines support to one of my more simple projects and unfortunately found it extremely tedious to have to push to Bitbucket every time to see if I fixed the build.  So, like any good Open Source developer, I started working on a solution.

Pipelines is built on top of Docker and uses a YAML file to describe how the build works.  I've been using Docker for very long time now (I gave a talk on it in August 2014 for anyone that's curious) so I'm very comfortable with it.  That said, all I really needed to do was take the YAML file and turn it into some docker run commands.

So after a few hours of work I had a working version of what I later named local-pipelines.

It sat that way for awhile until Sean Farley ran into the same issues I did with not being able to test until pushing.  He then proceeded to clean the VCS interaction code and added support for passing environment variables into the pipeline.  His work got me working on it again too so I finally cleaned up the documentation and got it uploaded to pypi.

You can find more information on the overview page or if you're using MacPorts you can find it there as well.

Building libpurple3 on OSX with homebrew

I've spent a far amount of time this week working on making it easier (or even possible in some cases) to build libpurple3 on OSX.  I haven't gotten to Gtk+ yet, so I haven't even tried compiling Pidgin yet.

Typically OSX isn't one of first party supported platforms, but I'm on vacation this week and only have a MacBook at my immediate disposal.  So here we are ;)

Most of the issues have been in the generation of the configure script and landed in PR #103.  But there was a side effect from an earlier PR that became a problem on OSX since homebrew does not have farstream.  PR #107 has been submitted to fix that.

When PR #107 is merged everything should be buildable.  But it does not and will not work out of the box.  Why you ask? Politics!  (of course...)

Homebrew attempts to play things very safe, which is admirable, but a giant pain when you're actually trying to compile something that isn't already in Homebrew.  So we go about making this easier by using a little known feature of Pidgin's build system.

Years ago I got tired of trying to remember what arguments I was passing to autogen.sh and configure.  So like any programmer, I added code that would do it for me!  That code sources a shell script named autogen.args from the directory you invoke autogen.sh from.  So in the root of my build directory I have a file named autogen.args with the following content:

export PKG_CONFIG_PATH=$(brew --prefix libffi)/lib/pkgconfig:$(brew --prefix libxml2)/lib/pkgconfig
CONFIGURE_FLAGS="--disable-gtkui --disable-consoleui --disable-vv --disable-kwallet --disable-meanwhile --disable-avahi --disable-dbus --disable-gnome-keyring --enable-introspection"
As you can see, I'm disabling a ton of stuff to make this build work, but that's not all.  Notice the PKG_CONFIG_PATH environment variable being exported.  This is dealing with Homebrew's policy of staying out of the system's way.  OSX has it's own version of libffi and libxml2 so Homebrew will not install any part of those packages where the system will look for them.  This has the unfortunate side effect of causing weird build failures until you realize this is the problem.

At any rate, I hope this has been helpful to someone :)

Tuesday, March 17, 2015

Chromecast, Android, UPnP/DLNA, and Docker

If you're like me, you have at least one machine on your home network that has a UPnP/DLNA server up and running it on.  You may have used a PS3, XBOX360, NeoTV, or any other UPnP/DLNA device to watch media from that UPnP/DLNA server.  And... Like me, you may have grown tired of all these machines and tried to migrate everything to using a Chromecast.  If so, you soon discovered Chromecast has no way to natively play anything via UPnP/DLNA.

This is the situation I found myself in.  I searched the Play store and found BubbleUPnP.  It sounded great, then I tried to use it.  Well by default it can only cast to your Chromecast in the native formats that your Chromecast supports, which as I'm sure is not what the majority of your media is encoded in.

Luckily, BubbleUPnP has a server that will transcode media from whatever format it is in into a format that your Chromecast supports.  The Android app will even automatically find the server for you automatically.  However, the server is kind of tricky to setup and get working.  Fortunately I've been playing with Docker a lot lately.

So if you're trying to cast UPnP/DLNA and you happen to be running a Linux distribution that's capable of running Docker, you can just run the Docker image I've created for the BubbleUPnPServer.

There were existing Docker images of the BubbleUPnPServer, but they all had something weird about them which you can read in more detail at the official page for the image.  But to get you started you just need to run the following command on a machine with Docker installed on your local network.

    docker run -P --net=host rwgrim/docker-bubbleupnpserver

Hope you find this as useful as I have!

Thursday, January 9, 2014

Long time no post...

I've been super busy as of late with lots of projects and other stuff, but figured a blog post is long overdue.

As some of you may know, there was a Pidgin Summer of Code project for GObjectification.  As some of you may know, I designed and spec'd out a ton of that project.  I was not the primary mentor for the project, but more of a technical support.  Anyways, when the topic of plugins came up, my GPlugin library got chosen.

My last post on GPlugin was announcing version 0.0.2 in April of 2012!!  Anyways, over the summer, GPlugin's functionality grew leaps and bounds.  The Python loader was finished, a Perl loader was started, a Lua load was completed and the library itself is more or less feature complete.  There's still plenty of things that need to be done, but everything (aside from a bug or two) is completely ready to use!

Last night I released version 0.0.12 which pretty much just fixed an extreme corner case bug that was more of a memory leak and some more unit testing.  At any rate, if you're working on a Glib/GObject based application and want plugins, give GPlugin a go as I could really use some more feedback!

Sunday, April 29, 2012

gplugin 0.0.1^H2 released!

After *MANY* months of on again off again coding I'm very proud to announce version 0.0.2 of GPlugin.

Version 0.0.1 had a broken pkg-config file, which I fixed in 0.0.2... I knew I was forgetting to test something...

GPlugin is a library that will give your program GObject based plugins.  Right now it only supports native (compiled plugins), but there are plans for at least a python loader and whatever else anyone wants to write.

During this release I transitioned this project off of guifications.org to bitbucket.  All GPlugin related business (bug reporting, file downloads, etc) should be done at the bitbucket site.

Also, documentation is in the source and will eventually be built/posted once the gobject-introspection guys figure out whats going on with g-ir-doctool.  Right now you can build the docs and open them in yelp, but not devhelp. I'll put something on the bitbucket site soon for how to do that.

Source tarballs can be found here: zip gz bz2

Tuesday, March 13, 2012

nodm and matchbox window manager and auto starting applications

I'm working on a project for work that is based around a live distro.  For the live distro building, I went with live-build since Debian is by far my favorite distro.  The live build was pretty easy to implement, and I'm not going to bore you with the details when there's plenty of documentation out there, especially the live-manual.  This post is all about how to get Xorg started and how to launch an application when Xorg starts up without diverting files in other packages, or having a match install anything to a users home directory.

The clear start to this task is nodm.  It'll automatically login as a user and start your xsession.  If you don't have an xsession installed, it'll fall back to the normal Debian xsession script that'll just launch an xterm.  This is simple enough, but remember that without a window manager, you're not going to get any of the standard desktop magic that the freedesktop.org window manager specification gives you.  For example, being able to call gtk_window_maximize.

To fix that problem I went with matchbox window manager.  It's very light weight window manager meant mostly for mobile devices and runs apps in full screen, thus eliminating my need to maximize the application window.  Matchbox registers itself as an alternative for x-window-manager under Debian which, in this bare bones approach will start matchbox on boot, but no apps.

Luckily, the matchbox package depends on matchbox-common and matchbox-common contains /usr/bin/matchbox-session.  This script looks for a session script and will execute it, if it doesn't find one, it'll launch matchbox-desktop and matchbox-panel in the background, and then launch matchbox-window-manager in the foreground.  The real awesomeness here, is that matchbox-session checks if there is an executable script named session in /etc/matchbox/.  What this means, is that you can easily create a package that depends on matchbox and install that file.  This fulfills all of our requirements!!  But wait, matchbox-session won't be called from nodm...

Solving that problem is actually a lot easier than you're probably thinking.  Debian's Xsession scripts give a higher priority to x-session-manager which is in the alternatives system.  So all we need to do, is to register matchbox-session as an alternative for x-session-manager.  Once this is done, our custom /etc/matchbox/session will get executed and we're good to go :)

Due to NDA's and stuff, I can't post my exact package here, but here's the jist of it.

Make sure your control file depends on "matchbox" (that's the meta package for matchbox and will pull everything in); you can probably clean it up to just matchbox-common and matchbox-window-manager, but I leave that to you.

Next create a postinst script that contains the following:

#!/bin/sh
set -e
case "$1" in
        configure)
                update-alternatives --install /usr/bin/x-session-manager \
                        x-session-manager /usr/bin/matchbox-session 50
        ;;
esac
#DEBHELPER#
and a prerm which contains the following:

#!/bin/sh
set -e
case "$1" in
        remove)
                update-alternatives --remove x-session-manager /usr/bin/matchbox-session
        ;;
esac
#DEBHELPER#
Then create your custom session script.  Mine looks something like this:

#!/bin/sh
application &
exec matchbox-window-manager
Make sure all three of those are executable, add the session script to your install file, fire off debuild  and you should be good to go.

I hope this helps other, since googling for this earlier came up pretty empty :)