Ahmed Jazzar's profile picture

Integrating third-party templating libraries with Django

As web developers, we all understand the importance of having a web framework that provides decent solutions for generating HTML pages dynamically using server-side scripting. Django followed the most common approach that relies on templates by introducing to us Django Template Language (DTL) which uses a new templating syntax. Also, Django natively supports the popular alternative Jinja2 that's well known for being faster in render and efficient in using memory than DTL. You also have the ability to integrate one or several Python template libraries that are not natively supported in Django with your Django project.

Django picture

Last week as I'm working on integrating Mako templates with a Django project I'm working on, I noticed the lack of resources on how a Django developer can use Mako instead of (or alongside) DTL, so I felt like sharing my experience and the minimal solution I came up with to other developers who might be willing to support Mako templates in their projects.

In this post, you'll go through the detailed implementation of integrating your own template library (I used Mako for demonstration here) with your Django project. If you just want to see/use the final solution you can skip this post and jump to the Gist files I wrote here. All files are well documented to help you build a good sense of the problem while solving it.

However, if you're still reading this line, then you're going to find yourself getting into the boring explanation of everything related to integrating a new template library and making it works. I feel like mentioning here that you may not find enough explanations to some concepts or configurations as they are not very related to the setup process, so if you think you didn't understand anything, please comment or search them on Django's documentation as you're gonna find a detailed explanation to them there.

To get started, you need to install your template library requirements. For Mako, you only need to have the following in your requirements file.

mako==1.0.6

pinning mako version is not necessary.

Configuring

Maybe it's not the best thing to start with, but this is the entry point for templates backends and objects that we are going to go through later in order to be able to render different templates syntaxes in the project.

For Django to know templates backends you're are using, it provided you a configuration setting called TEMPLATES that's a list of configurations dictionaries, one for each engine. The default value for it is an empty list, which means that your project doesn't have templates at all.

Each configuration defines the BACKEND engine for this configuration, the backend OPTIONS, and templates' DIRS directories list belongs to this backend that goes inside DIRS. The following code snippet placed all the possible various configurations you can list in TEMPLATES setting.

# ...

TEMPLATES = [
    {
        'BACKEND': 'backends.mako.MakoTemplates',
        'NAME': 'mako',
        'DIRS': [
          os.path.join(BASE_DIR, 'templates/mako'),
        ],
    },
    {
        'BACKEND': 'django.template.backends.django.DjangoTemplates',
        'DIRS': [
            '/home/html/templates/lawrence.com',
            '/home/html/templates/default',
        ],
    },
    {
        'BACKEND': 'django.template.backends.jinja2.Jinja2',
        'DIRS': [os.path.join(BASE_DIR, 'templates/jinja2')],
        'OPTIONS': {
            # Any option that's specific for this backend.
        },
    },
]

# ...
  • The first item in TEMPLATES configures an example of how you can configure your own backend by supplying the backend engine (we will build it later) in a dotted Python path to the template backend API as specified below. Notice that we gave the backend a specific name that must be unique across all configured templates engines. Name is an identifier that allows selecting an engine for rendering. It defaults to the name of the module defining the engine class.
  • The second item configures DTL backend and tells it to look for templates only inside the absolute directories /home/html/templates/lawrence.com and /home/html/templates/default if any listed directory not found on the server, the server will start normally like you didn't define them ever.
  • The third item configures Jinja2 backend, the special things in this case are that the configuration uses another Django-built-in engine and passes OPTIONS to it. Options are Engine-specific settings which default to {}. These options will be passed as keyword arguments when initializing the template engine later.

Engine Selection

Saying that TEMPLATES is a list, you might thought of its ordering and if does it matter or not. This question leads us to know how Django selecting template engines to render a specific template if multiple engines are configured. When Django tries to render a template, it must choose an engine to do the rendering task, and for it to do so, there're multiple ways to determine the proper engine.

Note that Django doesn't really know how to select the proper engine if you didn't tell it what template belongs to what engine.

  • Explicitly selecting an engine is a way to explicitly specify what engine you want Django to render your template with for a specific view. Doing so will skip engine searching as the search will be restricted to the specified engine. This method is not always good for the following reasons:
    • It adds some inconvenient boilerplate regardless of the API that's chosen.
    • You've to specify the template engine for every view.
    • If you're including third-party Django applications, this may restrict you from replacing built-in templates with templates written in your engine.

To explicitly select an engine in function-based views, give the render function your engine name by using using argument like this:

def mako_view(request):
  render_to_string('home.html', {}, using='mako')

def django_view(request):
  render_to_string('home.html', {}, using='django')

For class-based views you can give the render function your engine name by defining the property template_engine in the class view.

class MakoView(View):
  template_engine = 'mako'

class DjangoView(View):
  template_engine = 'django'
  • Backend's specific path Here you're going to put each backend template in a specific folder and link each path with the Backend configuration. This is identical to what we did in the previous snippet with Jinja2 and Mako configurations. This method will lead to some complications in maintaining multiple versions of some base templates like base.html or layout.html for each template backend if you want each different template from different engines to extend these files.
  • Trial and error Django will iterate over the list of configured template engines and attempt to locate the template with each one of them until one succeeds. Yes, this means that ordering is important, so if you want for example to make Mako your first templating language, you need to insert it at the TEMPLATES list beginning.

The Engine

that understands the library

Every template library should be able to retrieve templates given their names and compile them given their code if the code has no syntactical errors. And here comes the engine's responsibility of communicating with both, the template library (Mako in our case) to perform these two basic functionalities and the Django template engine backend to which we are going to send the end-results to.

The Engine is basically a class gets some configurations and initializes the library environment responsible of performing the tasks. We are going to get these configurations from the template OPTIONS dictionary defined earlier in our settings.py file.

In our engine, we are going to use Mako library class TemplateLookup to make locating and dealing with other templates effortless so that you easily inherit or include another templates in you template file.

class MakoEngine(object):
    def __init__(self, **options):
        environment = options.pop(
            'environment', 'mako.lookup.TemplateLookup')
        Environment = import_string(environment)
        self.context_processors = options.pop('context_processors', [])
        self.lookup = Environment(**options)

    def get_template(self, name):
        return self.lookup.get_template(name)

    def from_string(self, template_code):
        return MakoTemplate(template_code, lookup=self.lookup)

In our code above, we defined a MakoEngine that initialized the context processors and the lookup environment. MakoEngine class has the two important methods. The get_template_method is basically using TemplateLookup to help us locating the template, while from_string is using MakoTemplate that represents a compiled template given the template code.

“Note that I imported MakoTemplate like this for differentiation purposes: from mako.template import Template as MakoTemplate”

The Template object

that renders requests

Template objects returned by backends must contain a render method that is responsible of rendering the template given a context (the views context) and a request. In the render method you can add some extra context vars that should be consistent between and available to all templates. Your template library should be able to render the template given a context and returns back to you the rendered output of the template as a string.

In the example below, we defined the Template class that initializes the template and defined the required render method. The render method updating the context by adding some some variables that all templates share like the static url, the csrf_input and the request. You can simply ignore updating your context but you have to import them in almost every template, so it'd be a good idea to add your frequent accessed context variables here.

class Template(object):
    def __init__(self, template):
        self.template = template

    def render(self, context=None, request=None):
        if context is None:
            context = {}

        if request is not None:
            context['request'] = request
            context['csrf_input'] = csrf_input_lazy(request)
            context['csrf_token'] = csrf_token_lazy(request)

            context['static'] = staticfiles_storage.url
            context['url'] = reverse

        return self.template.render(**context)

The Engine

that goes through Django Backend API

In order to use another template system in Django, you need to implement a custom template backend. The template backend has to inherit django.template.backends.base.BaseEngine or define app_dirname property and implement the following methods at least:

  • __init__(self, params)
  • from_string(self, template_code)
  • get_template(self, template_name)

This engine is basically the BACKEND value we wrote earlier in Mako TEMPLATES settings configurations. Django expects you to implement from_string and get_template so it can call each one of them when it's necessary. from_string will be called to get a template object from a template code which will request Mako template library to compile the template and may raise TemplateSyntaxException if it couldn't understand the syntax. However, get_template method also returns a template object but this time using a template name not code. This method may not be able to locate or compile the templates so it'll raise TemplateDoesNotExist and TemplateSyntaxError for each one of these cases respectively.

The MakoTemplates below is going to take the passed parameters and initializes its superior, and then giving a default values to the passed OPTIONS. MakoTemplates will make use of the previously built engine (that understands the library) to use it in getting and rendering templates.

The from_string is using the defined engine's from_string method and wrap it inside a Template object to make it able to render requests which raises SyntaxException in case mako spotted a syntax error, we're then going to catch this error and re-raise the conventional syntax error TemplateSyntaxError.

The get_template method will also call the defined engine's get_template method to locate the desired template from its name and wrap it inside a Template object to make it able to render requests. This method will possibly raise two exceptions, the TemplateLookupException which we're gonna catch it and raise it again as TemplateDoesNotExist exception and CompileException which is gonna be re-raised as TemplateSyntaxError.

class MakoTemplates(BaseEngine):
    app_dirname = 'mako'

    def __init__(self, params):
        params = params.copy()
        options = params.pop('OPTIONS').copy()
        super(MakoTemplates, self).__init__(params)

        options.setdefault('collection_size', 5000)
        options.setdefault('module_directory', tempfile.gettempdir())
        options.setdefault('output_encoding', 'utf-8')
        options.setdefault('input_encoding', 'utf-8')
        options.setdefault('encoding_errors', 'replace')
        options.setdefault('filesystem_checks', True)
        options.setdefault('directories', self.template_dirs)

        self.engine = MakoEngine(**options)

    def from_string(self, template_code):
        try:
            return Template(self.engine.from_string(template_code))
        except mako_exceptions.SyntaxException as exc:
            raise TemplateSyntaxError(exc.args)

    def get_template(self, template_name):
        try:
            return Template(self.engine.get_template(template_name))
        except mako_exceptions.TemplateLookupException as exc:
            raise TemplateDoesNotExist(exc.args)
        except mako_exceptions.CompileException as exc:
            raise TemplateSyntaxError(exc.args)

Django source code and documentation contains example code for a base engine which you can check it here.

Testing

The final step in this long-short-process is to test our changes so far. To do so, we are going to run the following shell management command

python manage.py shell

This will open the Django management console where we are going to test simple functionalities.

First, we are going to import the configured engines from django.template, and then test the engine's from_string in DTL for sanity check purposes only. You'll notice below how we fetched the Django template engine and passed a string template to it, then rendered it given some context.

from django.template import engines

django_engine = engines['django']
template = django_engine.from_string('Hello {{ name }}!')
context = {'name': 'Ahmed'}

template.render(context)
# >>> 'Hello Ahmed!'

We're going now to repeat the previous steps in the same shell console but for our mako engine.

from django.template import engines

mako_engine = engines['mako']
template = mako_engine.from_string('Hello ${name}!')
context = {'name': 'Ahmed'}

template.render(context)
# >>> 'Hello Ahmed!'

Notice how both engines (DTL and Mako) behaved perfectly as expected through Django default templates APIs without too effort!

Reaching this point, we've been able define our own template library's engine and to cover almost all important and necessary parts of integrating third-party templates engines with Django.

If you are still looking for a full one-pieced solution you can find it hosted on Github Gist here. Just place the code in the proper places in your project and every thing should work perfect.

Love,
Ahmed Jazzar