Life Beyond Apache: Setting Up Gunicorn on Ubuntu Lucid
Published on December 20, 2012
Recently it has come to my attention that there's a world of alternatives to the ubiquitous LAMP server stack that I have grown so accustomed to. I have growing number of Django websites in the wild, and while they've been running under Apache and mod_wsgi to this point, my interest in faster alternatives has lead me to check out Gunicorn (Green Unicorn).
Gunicorn is very appealing because it is all python. Also, mod_wsgi has always kind of mystified me. But that is beside the point. Proponents of gunicorn talk about speed, small resource consumption, and overall stability. I decided to migrate a handful of Django projects off of apache and onto Gunicorn and Nginx (more about Nginx in the next post)
Implementing gunicorn is reasonably simple for the benefits it provides. Here's how I did it on an Ubuntu 10.04 (Lucid) system. First thing you'll need to do is hop into your virtual environment and
pip install gunicorn
Add the app to your settings file:
INSTALLED_APPS = (
....
'gunicorn',
....
)
From here, you can fire up the gunicorn server with a simple:
python manage.py run_gunicorn
If that works, you're on the right track.
Note you can also start the server via
gunicorn_dango
as long as you are in the virtual environment that you installed gunicorn in.
There are a wide variety of options to specify, but let's start with the number of workers, and the address and port to bind to.
something like this:
gunicorn_django -w 3 -b 127.0.0.1:8003
Give this a whirl and see if it works. (Gunicorn should report a few lines to the terminal if successful) If it works, you're ready for some automation.
Roll this command into a shell script. I've called it gunicorn.sh, and placed it in a conf directory within my project directory:
#!/bin/bash
set -e
LOGFILE=/path/to/my/gunicorn.log
LOGDIR=$(dirname $LOGFILE)
NUM_WORKERS=3
# user/group to run as
USER=admin
GROUP=admin
#move into project directory
cd /path/to/django/dir
#activate the appropriate virtualenv
source /home/admin/envs/myenv/bin/activate
#if the log dir doesn't exist, make it
test -d $LOGDIR || mkdir -p $LOGDIR
#summon the unicorns
exec /home/admin/envs/myenv/bin/gunicorn_django -w $NUM_WORKERS -b 127.0.0.1:8003\
--user=$USER --group=$GROUP --log-level=debug \
--log-file=$LOGFILE 2>>$LOGFILE
Make sure the shell script is executable, and test it as you did the basic command. Note you won't see gunicorn's log output in the terminal as you did when running the basic command -instead check the log file you specified.
Now we have a single command to start the server, but it is still a manual process, including restarting it when and if it crashes. This may be fine for active development, but if you'll need to automate these tasks for staging or production. There are several ways to do this- two popular choices are Supervisor and/or Upstart. I've ended up using them both.
First we need to install Supervisor and tell it to hit the shell script. To install,
sudo pip install supervisor
Refer to the Supervisor docs for more reference, but once installed, run
echo_supervisord_conf
If this returns a config file in your terminal, you've got supervisor installed properly. Next, dump this output into a real config file: (as root)
echo_supervisord_conf > /etc/supervisord.conf
edit supervisord.conf:
sudo vim /etc/supervisor/supervisord.conf
add this at the end:
[program:myprogram]
directory = /path/to/django/dir
user = admin
;start this program when supervisor starts:
autostart=true
;respawn processes if it dies
autorestart=true
;the shell script that runs your gunicorn instance
command = /path/to/my/shellscript/conf/gunicorn_live_dav.sh
stdout_logfile = /path/to/logs/super/stdout.log
stderr_logfile = /path/to/logs/super/stderr.log
The key is the command line- this runs the shell script that you created to start the gunicorn server. Add additional programs for each site/application you'd like supervisor to control.
Start up supervisor via:
/usr/local/bin/supervisord -c /etc/supervisord.conf
Check your running processes (ps aux) to see if supervisor started and successfully ran your program. At this point, you can also jump into supervisor's interactive manager via
sudo supervisorctl
this should report the status of your progam, including the PID and uptime. from here, you can start/stop/restart your program by name.
So now we've got a convenient mechanism to control your gunicorn processes. This is especially helpful if you have several instances of gunicorn running on a server, and are using them to serve Django applications to several different websites. However, supervisor still needs to be started manually. If your server experiences a reboot, it would be so very nice to start supervisor at startup. This is where Upstart comes in.
I've decided to call my Upstart job "super" and I can create it like this:
sudo vim /etc/init/super.conf
per Jason Cupp's advice, I've avoided the names supervisor.conf or supervisord.conf.
description "Start Supervisor"
start on runlevel [2345]
stop on runlevel [!2345]
respawn
exec /usr/local/bin/supervisord --nodaemon -c /etc/supervisord.conf >/tmp/super_upstart.out 2>&1
note: adding >/tmp/super_upstart.out 2>&1 at the end of the exec command provides a nice way to log any output that supervisor may burp out. This is especially helpful if something goes awry. It is not necessary for production.
Now supervisor will start on system boot (run level 2-5), and you can control it via:
sudo status super
sudo stop super
sudo start super
Give the whole shebang a restart to make sure it works, and if you've been successful to this point, it is time to move on to the next layer in the stack: in my case, Nginx.