hasenj blog

The problem with Django

Posted in webdev by hasen on 11/04/2009

Frameworks speed your development time at first, but then you often hit a wall. The framework doesn’t do what you want and you have to dig into its internals to figure out how to do what you want, which is often more difficult than writing your own framework from the ground up. — Cal Henderson, Flickr’s Engineering Manager, at a django conference

In many ways Django reminds me a bit of unpacking cool stuff. You download it and are instantly blown away how cool it is. Database just works, templating is awesome, URL routing gives you these incredible neat looking URLs and much more. It’s so cool that many are trying to suddenly do everything with it. And that’s kinda where the problem lays. You are forcing Django to do things it’s not designed for. — Armin Ronacher, in a blog post

I’ve been using Django for about, almost two years, and I came to the same conclusion. Django is amazingly awesome at first, but not so awesome in the long run.

The reason Django is so amazing at first is that it relieves you from having to do the stupid repetitive tasks involved in writing a typical php app: writing an admin interface, writing a class that maps to your db table, and doing all the mappings manually. I’ve had little experience writing a mini-blog like website using php, and it was an exciting experience to build my first web-app ever (it was a school project). However, even though it was exciting, I also didn’t like the thought of having to repeat all that redundant work all the time, so in that regard finding Django was a big relief.

However this awesomeness doesn’t last so long. Soon enough the requirement get complicated, and Django doesn’t wanna do what you want it to do anymore. You’re forced to either wrestle with it, or implement the requirments completely outside django, which kinda defeats the point.

It’s a Framework!

See, one of Django’s problems is that it tries to do everything for you, forcing you in the meantime to adapt to its way of doing things, instead of it adapting itself to you.

Django has an (awesome, btw) automatically generated admin site, and as awesome and incridible as it is, it’s suitable only for data entry. It relieves you from the initial effort of writing the admin interface. You can get started with a simple application by just specifying the models and writing a couple of simple templates, without ever worrying about SQL or writing (an) Admin page(s) for entering you data. But is that really so great? I mean, it gives you a relief in that you don’t have to do it, but at the same time, it’s so simplistic, it’s not the kind of thing that’s suitable for all situations; infact it’s quiet unsuitable for most situations.

I’m writing this blog post right now in wordpress. Have you seen the admin interface? It’s so great, but .. it’s all written in php! Django’s admin interface can never ever come close to this. No matter how much you try to customize it (and there isn’t that much to customize anyway). Of course you can build one from scratch, (in python), but then what’s the point of doing this inside a Django framework? See, when you start doing things outside Django, you realize that you don’t even need django. Not only that, but Django will get in your way because you can’t just do what yo want, you have to do it in a way that pleases Django.

Remember the first quote above from Flicker’s manager? It speeds you up at first, but then you hit a wall and you have to start from scratch again.

I’ve done an ajax admin interface to some application (that shall remain anonymous), and I didn’t use one bit of Django’s forms framework. Why? Because I don’t need to. It’s much easier to write my own forms in html, and my own validation functions, than to constrain my self to the framework.

I kinda forgot t mention this but (as you might have guessed already): Django has a forms framework. I’ve never looked deeply into it, but I can tell it suffers the same issues as the admin interface: it make so many assumptions about your workflaw, and forces you to adapt to /its/ workflow, instead of it adapting itself to yours. Plus, forms are relatively simple to write in plain html (and much easier to customize when you write them that way), why should I spend time to learn a complicated Forms framework? Is it worth it? I doubt that.

The same thing can be said about the template system, and the crazy idea that you have to write a costum tag (read: lexer and parser) for every extra little thing you need. Thank God there’s Jinja!

Anyway I don’t think I need to talk about the pitfalls of Django’s template system much, it’s kinda obvious for anyone who’ve used it. I will just mention that I had *so much pollution* in my python code to cover up for the shortcoming of the template system, and even after I switched to jinja, some of that poison is still lurking somewhere, in some deep dark rusty corners of my code.

Of course, the auth and session middlewares suffer from the same thing: they’re awesome, but not really extensible. At least in the sense that it’d be much easier to roll your own solution from scratch than wrestling with the framework and digging into its internals.

Fragmentation

After a while of working on a website with several “apps”, you come to realize what a mess everything is. Everythign is torn apart. You can view the application as being composed of 3 basic parts:

  • python code
  • template files
  • media files

The media files can be broken down further into: images, javascript, and css; but let’s not get into that, as it’s outside the realm of Django.

The problem is that each of these components lives in a completely different planet. You have a directory for code, a directory for templates, a directory for media. Each directory is in a completely different place from the rest. When you need to make changes to an application, *you have to remember quite a lot of things*, if you *only* remember where each directory was, you’re already using 3 slots in your short term memory. It might not seem like much, but it does get daunting after a while. It becomes a wall that makes you go “oh no, I have to go through that again?”, pushing you to procrastinate, thus making you less productive.

Furthermore, the python code has to be further fragmented according to the specifications and expectations of the framework. You need a urls.py file, models.py, admin.py, views.py (and maybe you tag along with this idiom and create a forms.py, and maybe utils.py, or biznis_logic.py, or whatever)

So if your app has four components (in terms of models), then different pieces that deal with different aspects of each component are fragmented across these files.

Many kinds of changes to the code will require “shotgun surgery”; lots of little changes to lots of files.

This fragmentation is caused by the framework; because it *forces you* to layout all of your files in a certain way; it forces you to adapt to *its* work-flow.

What’s left?

So, so far I’ve dismissed the template engine, the forms framework, the admin interface, and the file layout.

What’s still remaining? The url-mapping, and the ORM.

Well, I recently learned about Werkzeug, so I don’t have to use Django for url-mapping. As for the ORM, there’s SQLAlchemy.

The thing that’s attracting me to Werkzeug is that it (by design) is anti-framework. I haven’t used it for real yet, but that’s the kind of impression it’s giving me.

As for the ORM, I think Django’s ORM is great, but if SQLAlchemy also solves the same problem, then I don’t have to stick to Django just for the ORM. Plus, SQLAlchemy is dedicated to being an ORM, while Django’s ORM is a part of a larger project, therefor I would assume that SQLAlchemy has a better chance of doing things right.

Things I’d like to keep from Django: the pagination module, and the email module; as python’s smtplib just sucks!

Tagged with: , ,

Integrating Jinja2 with django

Posted in webdev by hasen on 12/11/2008

(I have this posted on django snippets: http://www.djangosnippets.org/snippets/1112/ and it’s where you can view the code and download it)

Jinja2 is an alternative template system that can be plugged into django. It offers greator flexibility in presentation logic; if you find the django template system too restrictive, you should have a look at Jinja2 (The syntax is very similar).

In Jinja, you don’t need costum tags (most of the time), because you can call functions and pass them parameters too! The only problem is that you cannot “load” functions from the template, this “loading” must be done by the code that renders the template. Same goes for filters and tests.

If you need only two or three functions/filters, no problem, you can manually add them to the Environment object; but this method doesn’t really scale with real-life webapps.

This module/snippet offers a solution by allowing the user to define application-specific functions and filters and load them into the jinja2 environment automatically.
It also sets auto_escaping on (Jinja2 has it off by default).
It searches for templates in TEMPLATE_DIRS and also in the “templates” subfolder (if it exists) of each application folder.

The approach I’ve taken to specify functions and filters is like this:

To add test functions, put them in a file called “jtests.py” and let each function start with “is_test_name”; it will be available as “test_name” in your jinja2 environment, so you can test {% if var is test_name %}

The same idea applies for filters, put them in a file called “jfilters.py” and let each one start with “filter_name”, and it will be available as “name” in the environment, so you can say “{{var|name}}”

For functions, it’s a little different; you add them a file called “jtemp.py” (temp is my wacky abbreviation of template global functions), but they are put in a name space that’s named after the application. so if your application is called “app”, then any function you put in jtemp.py will be avaiable in the template environment as {{ app.function_name() }}

Here’s how to use this:

  1. Install Jinja2 (for a quick & dirty installation, you can just put the jinja2 folder in a folder that’s in PYTHONPATH)
  2. Save this python module somewhere and call it something meaningful (I have it as jinja.py in my project directory)
  3. Whenever you need to respond (an HttpResponse) with a template that’s rendered by jinja, import this module and call return jrespond( template_name, context )
  4. Any filters or functions you define in any of your applications will be readily available inside the template (see the documentation in code)
Tagged with: ,