Sunday 29 December 2013

Installing Flower on Webfaction

Flower is a nifty little tool for getting visibility into a Celery queue, this is especially handy if ( like me ) you’re using Redis as a broker, which unlike RabbitMQ, doesn’t have a spiffing GUI.

In this post I’m going to demonstrate how I control flower with supervisor and expose it to the web via an SSL enabled nginx virtualhost that allows access for a small set of IP addresses.

Note that it is quicker and easier to expose flower directly to the web and enable basic auth, but that doesn’t float my boat. I feel better having it behind a reverse proxy with SSL enabled and having the process managed by something solid because this is a shared server and it is the wild west.

TL;DR.

This is the long way round, if all you want to do is access flower over the web via HTTP using basic auth, then ignore this post. If however you want to access flower via an HTTPS reverse proxy that is locked down via IP and manage flower using supervisor, then read on.

Things you’ll need.

This all assumes that you’ve got Nginx installed and configured and compiled with the real-ip-module along with supervisord & supervisorctl and virtualenv If you haven’t then you may find this a bumpy ride.

Create dirs, virtualenv and install flower.

I’m installing flower into it’s own virtualenv to keep things nice and separated.

cd sites
mkdir flower
cd flower
virtualenv venv
. ./venv/bin/activate
pip install flower

Claim your port.

one does not simply expect to use default ports

Boromir is right, you cannot simply expect to use default ports on Webfaction. Even if they work to begin with, I guarantee they’ll fail as soon as you stop watching your application. So head over to the control panel and create a new custom application that listens on a port.

Make a note of the port number that you get assigned.

Caution: Webfaction gives you the option of exposing your custom application to the world; don’t do this. But if you want to [whack a bean up your nose][10][^nose] and make flower directly available to the world, be my guest.

Start up the flower process via supervisord.

I’ve got supervisor installed and running[^circus] so I’m going to use that to control the flower process. Here’s my config file into which I’ve hammered the variables you’ll need to change for your context.

[program:flower]
command=/home/$YOURUSERNAME/sites/flower/venv/bin/celery flower --port=$YOURPORT --broker=redis://:$REDISPASS@127.0.0.1:$REDISPORT/$REDISDB
directory=/home/$YOURUSERNAME/sites/flower
user=$YOURUSERNAME
autostart=true
autorestart=true
redirect_stderr=true

Once that’s saved into a place that supervisor expects to find config files ( ~/etc/supervisor/ is my place of choice ) then you need to get supervisor to read it and add it as a process which is all done via the lovely supervisorctl shell.

supervisorctl
supervisor> reread
flower: available
supervisor> add flower
flower: added process group
supervisor> status
flower                           RUNNING        pid 550518,     uptime 0:00:02
#boom

Groovy, we’ve got flower up and running, but as of yet there is no way to get to it. Unless of course [you rammed a bean up your nose][10].

Nginx config

As I said at the start of this post, I prefer to lock things down via IP as it’s one less thing to manage and I have a number of static IP addresses at hand to me.

Before you proceed you’ll need to create an SSL website via the Webfaction control panel and configure your custom Nginx to listen on the right ports[^ssl].

Here’s my config, as with the supervisor example above, I’ve put variables into the places that you need to configure for yourself –


# the main app proxy - this should be the SSL enabled host

server {
    listen $YOUR_SSL_PORT;
    server_name $YOUR_DOMAIN;
    root /home/$YOUR_USERNAME/sites/flower/;

    location / {
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-Proto https;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_pass http://127.0.0.1:$YOUR_FLOWER_PORT;
    add_header Pragma "no-cache";
    client_max_body_size 1m;
    set_real_ip_from 127.0.0.1; 
    real_ip_header     X-Forwarded-For;
    allow $YOUR_STATIC_IP; 
    allow $ANOTHER_STATIC_IP;
    deny all;
    break;
    }
}

server{
    # redirect any non-ssl request to the ssl virtualhost
    listen $YOUR_NONSSL_PORT;
    server_name $YOUR_DOMAIN;
    rewrite  ^/(.*)$  https://$YOUR_DOMAIN/$1  permanent;
}

[^nose]: You should make the time to read this [excellent post by Jared Spool about sticking beans up your nose][ [^circus]us]: Maybe one day I’ll love Circus as much [^ssl]: Configuring SSL with Webfaction can be a royal pain in the arse, but they do try to make it as painless as possible.

[10]: http://www.uie.com/brainsparks/2011/07/08/beans-and-noses/?msg=hey Jared, I think you’re ace. We should have coffee from @jamiecurle