Django Async

Building a database backed task queue is a fairly trivial thing, but getting the database transactions exactly right is no simple matter.

Installing Django Async

Get Django Async with pip from pypi:

pip install django-async

Installation is very simple, just add the async application to your Django applications in settings.py. You also really want to use the transaction middleware (see below) and a proper transactional database (like PostgreSQL).

Using Django Async from your code

To run a job asynchronously just use the schedule function:

from async import schedule
schedule('my.function', args=(1, 2, 3), kwargs=dict(key='value'))

Tasks can be run by executing the management command flush_queue:

python manage.py flush_queue

See Using Django Async from your code for more details.

Using Django Async from the command line

Command line interaction with Django Async is through Django's management commands.

python manage.py flush_queue

For more details on options see Using Django Async from the command line.

Transaction handling

Database transactions are hard to get right, and unfortunately Django doesn't make them much easier. Firstly, you really want to be using a proper transactional database system.

Django has two major flaws when it comes to transaction handling:

1. The Django transaction functionality fails to create composable transactions.
2. The Django documentation makes a very poor recommendation about where to put the django.middleware.transaction.TransactionMiddleware.

The first problem is not going to get fixed in Django, but the second can be handled by putting the middleware in the right place — that is, as early as possible. The only middleware that should run before the transaction middleware is any whose functionality relies on it being first.

Within the async task execution each task is executed decorated by django.db.transaction.commit_on_success. This means that you cannot execute a task directly from within a page request if you are using the transaction middleware (this is due to problem number one above).

Pages

1. Using Django Async from your code
2. Using Django Async from the command line