Kitchen & dependencies

Imagine briefly that you have a superb Django project like, say Croisé dans le métro . You may have these problems :

There is a solution, and here it is step by step :

Pip

Pip is an easy_install on steroids, that will allow us to describe our project’s dependencies and will take care of install them for us. But first we need to install it :

$ sudo easy_install pip

Next, head up to the root of your project, we assume that it already contains a directory which contains your’re whole project. Just create a directory named deploy that will contain files describing our dependencies for each deploy.

$ mkdir deploy $ touch production.txt development.txt

The idea is that every file describe every dependencies needed to run our project in the corresponding environment :

$ nano production.txt ipython==0.9.1 python-memcached>=1.44 perlinpinpin>=0.1 -e git://github.com/brosner/django-announcements.git@master#egg=announcements -e svn+http://code.djangoproject.com/svn/django/branches/releases/1.0.X#egg=django
$ nano development.txt ipython==0.9.1 perlinpinpin>=0.1 fabric>=0.9 -e git://github.com/brosner/django-announcements.git@master#egg=announcements -e svn+http://code.djangoproject.com/svn/django/branches/releases/1.0.X#egg=django

We have just described our basics dependencies, by using package name or a specific version for a package. We can also provide subversion, git, mercurial and bazaar repository to pip, you can precise branch or revision easily too, just take a look at pip documentation for further details.

Virtualenv

Now that we can describe our project’s dependencies, we are going to create a virtual environment (for python stuff only) to isolate us and liberate us from our OS python environment.

But first thing first, install :

$ sudo pip install virtualenv Downloading/unpacking virtualenv Downloading virtualenv-1.3.3.tar.gz (1.0Mb): 1.0Mb downloaded Running setup.py egg_info for package virtualenv Installing collected packages: virtualenv Running setup.py install for virtualenv Installing virtualenv script to /Users/tim/Projects/app/env/bin Successfully installed virtualenv

Then we need to initiliaze our virtual environment at the root of our project in a directory named env :

$ virtualenv --clear --no-site-packages env Not deleting env/bin New python executable in env/bin/python Installing setuptools............done.

For those who would like to specify where their python is hiding :

$ virtualenv --clear --no-site-packages env -p /opt/local/bin/python2.6 Running virtualenv with interpreter /opt/local/bin/python2.6 Not deleting env/bin New python executable in env/bin/python Installing setuptools...........done.

Now that every thing is set up, we just need to install the dependencies described above :

$ pip install -E env -r deploy/development.txt [..] Successfully installed

If our virtualenv environment don’t exists, pip will create it for you (as long as virtualenv is installed).

Our dependencies are now installed, we need to activate our virtual environment to use them :

$ source env/bin/activate (env) $ python Python 2.5.1 (r251:54863, Feb 6 2009, 19:02:12) [GCC 4.0.1 (Apple Inc. build 5465)] on darwin Type “help”, “copyright”, “credits” or “license” for more information. >>> import django >>> django.file’/Users/tim/Projects/app/env/src/django/django/init.pyc’ >>>
$ source env/bin/activate (env) $ python app/manage.py runserver

You can add more dependencies if you like to :

$ pip install -E env “markdown>=2.0.1” Downloading/unpacking markdown>=2.0.1 Downloading Markdown-2.0.1.tar.gz (71Kb): 71Kb downloaded Running setup.py egg_info for package markdown Installing collected packages: markdown Running setup.py install for markdown changing mode of build/scripts-2.5/markdown from 664 to 775 changing mode of /Users/tim/Projects/app/env/bin/markdown to 775 Successfully installed markdown>=2.0.1

You will be able to ‘freeze’ your environment with pip in order to create a new dependencies file corresponding to your new needs :

$ pip freeze -E env -r deploy/development.txt deploy/requirements.txt

Virtualenv & WSGI

The secret for using our virtual environment with apache and mod_wsgi is this little wsgi handler :

$ nano site.py import os, sys, site site.addsitedir('/var/www/app.com/current/env/lib/python2.5/site-packages') sys.path.append('/var/www/app.com/current') sys.path.append('/var/www/app.com/current/env/lib/python2.5/site-packages') os.environ['DJANGO_SETTINGS_MODULE'] = 'app.settings' import django.core.handlers.wsgi application = django.core.handlers.wsgi.WSGIHandler()

There no need to edit mod_wsgi configuration, but you might want to specify the new path of your python through the WSGIPythonHome directive :

WSGIPythonHome /var/www/app.com/current/env

Fabric

Fabric will help us to deploy our application to our production or staging by using pip and virtualenv. If you have not read the first part of this article, fabric was in our dependencies (in development.txt) so you should be able tu use it within our virtual environment.

(env)$ fab Usage: fab [options] [:arg1,arg2=val2,host=foo,hosts='h1;h2',...] ...

The whole magic behind Fabric is in the fabfile.py file that live in the root directory of our project :

$ touch fabfile.py $ nano fabfile.py

Our fabfile describe several functions that produce an operation, most of the time we start by functions that set up our environments :

def production(): """Defines production environment""" env.user = "deploy" env.hosts = ['example.com',] env.base_dir = "/var/www" env.app_name = "app" env.domain_name = "app.example.com" env.domain_path = "%(base_dir)s/%(domain_name)s" % { 'base_dir':env.base_dir, 'domain_name':env.domain_name } env.current_path = "%(domain_path)s/current" % { 'domain_path':env.domain_path } env.releases_path = "%(domain_path)s/releases" % { 'domain_path':env.domain_path } env.shared_path = "%(domain_path)s/shared" % { 'domain_path':env.domain_path } env.git_clone = "git@github.com:example/app.git" env.env_file = "deploy/production.txt"
def staging(): """Defines staging environment""" env.user = "deploy" env.hosts = ['dev.example.com',] env.base_dir = "/var/www" env.app_name = "app" env.domain_name = "app.example.com" env.domain_path = "%(base_dir)s/%(domain_name)s" % { 'base_dir':env.base_dir, 'domain_name':env.domain_name } env.current_path = "%(domain_path)s/current" % { 'domain_path':env.domain_path } env.releases_path = "%(domain_path)s/releases" % { 'domain_path':env.domain_path } env.shared_path = "%(domain_path)s/shared" % { 'domain_path':env.domain_path } env.git_clone = "git@github.com:example/app.git" env.env_file = "deploy/production.txt"

Theses two functions might sounds a bit cryptic, but they are just defining a whole set of variables for each environment, that permits us to us the same action functions for every environment :

def start(): """Start the application servers""" sudo("/etc/init.d/apache2 start") def restart(): """Restarts your application""" sudo("/etc/init.d/apache2 force-reload") def stop(): """Stop the application servers""" sudo("/etc/init.d/apache2 stop") def permissions(): """Make the release group-writable""" sudo("chmod -R g+w %(domain_path)s" % { 'domain_path':env.domain_path }) sudo("chown -R www-data:www-data %(domain_path)s" % { 'domain_path':env.domain_path }) def setup(): """Prepares one or more servers for deployment""" run("mkdir -p %(domain_path)s/{releases,shared}" % { 'domain_path':env.domain_path }) run("mkdir -p %(shared_path)s/{system,log,index}" % { 'shared_path':env.shared_path }) permissions()

Those functions are called via the fab command. You just need to put name of each function in the order you wish to execute them. You can also call a function within another function like any other python function.

$ fab production setup restart $ fab staging setup restart

To make your life easier, this is a fabfile ready made, it’s loosely based on Capistrano), and offer refinements like :
– being able to rollback to previously deployed version (and environment).
– to launch South migrations.
– to push a maintenance page when deploying a new application, ut don’t forget to add those lines to your apache configuration :

RewriteEngine on RewriteCond /var/www/app.com/shared/system/maintenance.html -f RewriteRule !^/maintenance.html$ /maintenance.html [R=503,L] ErrorDocument 503 /maintenance.html

The deployment will take place in /var/www/${domain_name}/, your application will be accessible in the current directory et environment dépendatns files will be in the shared directory. Your will find your latest deploy in the releases directory. This fabfile deployment strategy is based on git, but using subverson or mercurial instead should not be a problem.

Download fabfile.py file.