Efficient component caching with Mako

Tags: mako cache controller component beaker

An example of how to compose efficient caching while maintaining a clear separation of code in the template and the controller. This recipe requires Beaker 1.5.

When creating a page that involves a hefty amount of logic and/or external services to be queried in addition to a complex or large HTML page, one generally ends up caching in two places.

The biggest problem with this pattern is that in the best-case scenario, where the HTML output is cached and the section of the controller for that portion is cached, there is still retrieval of the serialized objects in the controller for the template. This occurs even though the template output is still cached.

To remedy this, one can mark the section in the controller that should be run when the cached template output is no longer available, or when it should be considered invalid.

Setting up the Code

First, the necessary Python code, consider putting this in lib/component.py:

import beaker.cache
from beaker.util import func_namespace
from beaker.session import SHA1

def component_cached(region, *args):
    """Associate a callable with a cached page component.

    the page component should call the "render" def in
    conditional_cache.mako.

    The cache itself is local to the template in the render() call.

    """
    key = " ".join(str(x) for x in args)
    region = beaker.cache.cache_regions[region]

    def decorate(func):
        def cached(*args):
            cache_key = key + " " + " ".join(str(x) for x in args)

            def go():
                return func(*args)

            cache_rec = {
                'data_key':SHA1(func_namespace(func) + cache_key).hexdigest(),
                'region':region,
                'callable':go,
            }
            return cache_rec
        return cached
    return decorate

def invalidate_component_cached(source):
    """Takes the source information returned when calling a component_cached
    function, and marks it to be invalidated"""
    source['invalidate'] = True
    return source

Then create a Mako template called /component.mako:

<%def name="render(source)"><%
    cache_rec = source
    cachens = caller.context.get('self')

    def go():
        result = cache_rec['callable']()
        return capture(caller.body, **result)

    if cache_rec.get('invalidate', False):
        cache = cachens.cache.invalidate(cache_rec['data_key'])

    region = cache_rec['region'].copy()
    if 'expire' in region:
        region['expiretime'] = region.pop('expire')

    context.write(
        cachens.get_cached(cache_rec['data_key'], createfunc=go, **region)
    )
%></%def>

Using It

Inside a controller action, import the code and use it to indicate a section should be cached:

from datetime import datetime

from pylons import tmpl_context as c

from mako_sample.lib.base import BaseController, render
from mako_sample.lib.component import component_cached

class HelloController(BaseController):
    def index(self):
        @component_cached('short_term')
        def some_func(name):
            now = datetime.now()
            return dict(result='%s, the time is %s' % (name, now))

        c.func_src = some_func('fred')
        return render('/test.mako')

In this example, some_func will be cached. Repeat calls to the cached function where the output is changing should be assigned to another 'c' variable for use in the template. This variable is used to pass the reference to the function in so that the template can trigger it should its output be invalidated.

'short_term' here refers to a cache region. See the Beaker docs for more information on configuring cache regions in your Pylons application.

Now inside the template:

Hello, the cached data:

<%cache:render args="result" source="${c.func_src}">
The result is ${result}
</%cache:render>

<%namespace name="cache" file="/component.mako"/>

Some things to notice...

The cached function should return a dict, the keys of it will then be passed into the namespace render function in the template (this is why we're indicating in the template that it expects the args of 'result' back).

Second, while the triggering of the function happens from the template we still get the clean code separation of concerns by not having a ton of controller type logic inside the template itself.

Invalidating

To invalidate the cache, merely determine if it should be invalid in the controller, and use the invalidate function supplied:

from mako_sample.lib.component import component_cached, invalidate_component_cached

class HelloController(BaseController):
    def index(self):
        @component_cached('short_term')
        def some_func(name):
            now = datetime.now()
            return dict(result='%s, the time is %s' % (name, now))

        c.func_src = some_func('fred')
        if something:
            invalidate_component_cached(c.func_src)
        return render('/test.mako')

Comments (10)

densk

Sep 20, 2011 8:37:17 PM

Wow , there is really much useful data here!

pylonspylons

Oct 17, 2011 12:19:25 PM

Caching can cause issues and you have a workaround that appears to work. sim only contracts

star09

Jan 11, 2012 12:41:21 PM

More often than not I do not post on websites, but I would just like to say that this article has compelled me to do so! Thanks for your perceptive article.