Clabate: minimalistic class-based templates for Python

June 29, 2022

TLDR
Clabate does not offer yet another mini-language for templates. It is based on class hierarchy and PEP 3101 string formatting. Everything is declared in Python, natively. In the very basic layer Clabate implements bare textual templates which can be used, for example, to generate configuration files. And that basic layer is extended by MarkupTemplate, which escapes everything by default, trying to minimize chances to overlook unescaped substitutions:

from clabate import MarkupTemplate, Markup
from datetime import datetime

class HtmlPage(MarkupTemplate):
    html = Markup('''
        <html>
        <head>
            <title>{title}</title>
        </head>
        <body>
            <header>
                {header}
            </header>
            <main>
                {main}
            </main>
            <footer>
                {footer}
            </footer>
        </body>
        </html>
    ''')

class MyPage(HtmlPage):
    title = 'My web page'
    header = 'Today is {now:%Y-%m-%d}'
    main = '<<<Hello, world!>>>'
    footer = Markup('<span style="color:grey">Here we go!</span>')

    @property
    def now(self, context):
        return datetime.now()

my_page = MyPage()
context = my_page.render()
print(context.html)

➜ ➜ ➜

<html>
<head>
    <title>My web page</title>
</head>
<body>
    <header>
        Today is 2022-06-29
    </header>
    <main>
        &lt;&lt;&lt;Hello, world!&gt;&gt;&gt;
    </main>
    <footer>
        <span style="color:grey">Here we go!</span>
    </footer>
</body>
</html>
Web Technologies

Inception

I don't remember when I started to think "all that MVC is shit". A decade ago, at least. In terms of V, did you ever take a look at what your favorite template engine produces? I mean, at that auto-generated code which, in turn, generates the result? If you did that, did you ever think "I could do that much better by hands?"

Luckily, at my last work we accidentally used world-behated approach, placing all the HTML in Python code. We used %-based formatting with named substitutions. The approach was purely procedural, but our templates were hierarchical, well structured, and easily maintainable. In a tiny core routine we added some pre/post processing, just to make the output properly indented. Later on such a declassed approach seeded a thought in my head: could we do that even better?

In a variety of projects I often observed such things as header.tmpl, footer.tmpl, with all the stuff in between scattered across a bunch of files. The most recent case, by the way, happened just a year ago. It was Mercurial templates, when I embedded them into my declassed.art website using Clabate. With such a flat approach you can't even see at a glance if an overall HTML structure is correct. You have to look for opening and closing tags in different files.

Needless to say I don't like that. But I've got an idea: if the hierarchical approach perfectly matches class inheritance, so what if string templates were class attributes? They could be collected, dependencies could be analyzed, and then all those strings would be formatted in the right order?

That's how Clabate began. The initial version fit in about 200 lines of code. Later on I tried different approaches, added sugar and some extras, but this did not make Clabate too big:

File LOC SLOC
core.py 616 256
invoke.py 67 28
markup.py 370 147
extras/inclusion.py 63 23
extras/pygments.py 69 26
extras/sequence.py 68 20
examples/basic_examples.py 451 292
examples/comments_examples.py 33 19
examples/file_inclusion_examples.py 123 84
examples/invoke_examples.py 156 103
examples/markup_examples.py 396 281
examples/pygments_examples.py 52 39
examples/sequence_examples.py 39 27

Being in self isolation over the last decade I might miss the fact if someone already did such a thing. I don't care, programming is a kind of art and I code exclusively for sake of Art and Beauty. Also, more implementations is better.

Stop talking shit, give me another example!

Okay, here's some plain text example:

from clabate import Template
import time
from types import SimpleNamespace

class ZoneFileBoilerplate(Template):

    zone_config = '''
        $TTL    3600
        @   IN  SOA (
                    {primary_ns.hostname}.{idna_domain}.  ; MNAME
                    {rname}  ; RNAME
                    {timestamp}  ; SERIAL
                    3600  ; REFRESH
                    60    ; RETRY
                    1W    ; EXPIRY
                    60    ; MINIMUM Negative Cache TTL
                    )
        {nameservers}
        {resource_records}
    '''

    @property
    def timestamp(self, context):
        return int(time.time())

    @property
    def nameservers(self, context):
        ns_template = self.dedent('''
            @  IN  NS  {ns.hostname}.{idna_domain}.
            {ns.hostname}  IN  A  {ns.ipv4_addr}
        ''')
        result = []
        for ns in [self.primary_ns, self.secondary_ns]:
            result.append(self.render_str(context, ns_template, ns=ns))
        return ''.join(result)

    resource_records = '''
        @  IN  A   {main_server_ipv4}
        *  IN  A   {main_server_ipv4}
    '''

class DeclassedZone(ZoneFileBoilerplate):

    primary_ns = SimpleNamespace(hostname='ns1', ipv4_addr='1.2.3.4')
    secondary_ns = SimpleNamespace(hostname='ns2', ipv4_addr='5.6.7.8')


my_zone = DeclassedZone(idna_domain='declassed.art', rname='axy.{idna_domain}.')
context = my_zone.render(main_server_ipv4='9.10.11.12')
print(context.zone_config)

Output? Here it is:

$TTL    3600
@   IN  SOA (
            ns1.declassed.art.  ; MNAME
            axy.declassed.art.  ; RNAME
            1656230266  ; SERIAL
            3600  ; REFRESH
            60    ; RETRY
            1W    ; EXPIRY
            60    ; MINIMUM Negative Cache TTL
            )
@  IN  NS  ns1.declassed.art.
ns1  IN  A  1.2.3.4
@  IN  NS  ns2.declassed.art.
ns2  IN  A  5.6.7.8
@  IN  A   9.10.11.12
*  IN  A   9.10.11.12

I intentionally scattered strings across class attributes, constructor, and render calls just to demonstrate all the ways of parameterization. Note that strings passed to render and render_str are not treated as template strings. That's because all dependencies are collected in the constructor, when the class gets instantiated.

Dynamic content can be generated using class properties. Unlike traditional properties their getters have context argument. I tried to avoid this argument but that made use cases more cumbersome. You don't need to name that argument exactly as `context`. And if you plan to use such a property in usual way, make context a keyword argument with some default value.

Properties are evaluated once and the result goes to the context and reused when necessary.

What is context?

It's a rendering context, a subclass of dict. In particular, it implements __missing__ method that calls property getters and saves results they return.

Rendering context gets populated with all arguments you passed to the render, method, to the constructor of template, and with all formatted class attributes of template. Formatting applies to strings only. When rendering complete, you just pick the value of interest. There's no predefined name for the result.

How fast is Clabate?

I have no idea and have no desire to compare it with others. However, Clabate uses Python string formatting which is optimized quite well, I hope, and I simply rely on that fact. I know, any template engine should be very proud of its outstanding performance but I don't care. My goals were simplicity, pragmatism, ease of use, and nicely looking code. The only thing I could do was calling all potentially slow code from __init__. So, instantiate once and render many.

Clabate maintains proper indentation and this takes CPU cycles. If you're really concerned about performance and don't care about indentation, use LeanTemplate. In one of my particular cases it's 1.6x times faster than Template.

The use of the right base class is illustrated by markup templates: MarkupTemplate uses Template, thus maintaining indentation, but MinifiedMarkupTemplate uses LeanTemplate and simply shrinks spaces (along with JS and CSS minification).

So what's about markup, CSS and JS?

Minification depends on rjsmin and rcssmin, but these dependencies aren't mandatory.

However, the most important property of markup templates is the way of escaping. You don't have to explicitly say escape this, escape that, this substitution is a plain HTML and that substitution is a date... No. Clabate escapes everything by default unless you wrap your strings with Markup class. Previously this class was a simplified version of its namesake from markupsafe, but given its purpose in Clabate, it's no longer the case. I tried to keep things clear and this did not look achievable with confusing __html__ method.

An important exception is tag attributes. You should use {escape:attr('this is "<some value>", wow')}, or some_value = 'this is "<some value>", wow' and then {escape:attr({some_value!r})}, or, if you construct raw HTML in a property getter, self.escape.attr('this is "<some value>", wow'). Although, constructing raw HTML is not a good way, use self.render_str instead whenever possible.

Another important property of escaping is that Clabate never escapes strings twice. I know one way how to shoot in the foot and no doubt, as a newbie you'll do that in the first place, but for me, at least, it works as expected in most cases. I'll show you, take a look at this:

from clabate import MarkupTemplate, Markup

class SomeShit(MarkupTemplate):  # someshit sounds exactly as boxwood in my native language

    # instances of Markup class aren't escaped
    some_markup = Markup("""<span style="color:red">your HTML</span>""")

    # bare strings get escaped
    example1 = 'These characters will be escaped: <<<>>>, but {some_markup} will not.'

    # do you think the markup rendered in example1 will be escaped now?
    # haha, NO WAY!
    example2 = 'Example 2 includes example 1: {example1}'

    # maybe, it will be doubly escaped this time?
    example3 = 'Example 3 includes all previous examples: {example2}'

    # the last resort
    example4 = 'Example 4. WTF, at last??? {example3}'

boxwood = SomeShit()
from pprint import pprint
pprint(boxwood.render())

If you don't want to copy-paste and try running it, here is the output:

{'comment': <clabate.core.Comment object at 0x7f3f7e89af70>,
'escape': <clabate.markup.Escape object at 0x7f3f7e38fbb0>,
'example1': 'These characters will be escaped: &lt;&lt;&lt;&gt;&gt;&gt;, but '
            '<span style="color:red">your HTML</span> will not.',
'example2': 'Example 2 includes example 1: These characters will be escaped: '
            '&lt;&lt;&lt;&gt;&gt;&gt;, but <span style="color:red">your '
            'HTML</span> will not.',
'example3': 'Example 3 includes all previous examples: Example 2 includes '
            'example 1: These characters will be escaped: '
            '&lt;&lt;&lt;&gt;&gt;&gt;, but <span style="color:red">your '
            'HTML</span> will not.',
'example4': 'Example 4. WTF, at last??? Example 3 includes all previous '
            'examples: Example 2 includes example 1: These characters will be '
            'escaped: &lt;&lt;&lt;&gt;&gt;&gt;, but <span '
            'style="color:red">your HTML</span> will not.',
'some_markup': '<span style="color:red">your HTML</span>'}

Although context is a bit polluted with special names 'comment' and 'escape', this is my favorite piece of the whole artwork.

Hey, wait, what was that crap {escape:attr('this is "<some value>", wow')}???

Yes, it's a function call from template string which takes place at formatting stage.

Clabate provides Invoke class and escape is an instance of Invoke. This class has the only __format__ method and as you might already know, its argument, namely format_spec, is a string after the colon up to the closing brace.

Invoke evaluates that string as a Python expression. It's quite restrictive, it checks AST to ensure all arguments are simple values, not calls. Thus, you can't even use tuple() or dict() as arguments, you have to use (,) and {} respectively.

Extras contain some examples how Invoke can be used: include files, pygments, sequences. Just take a look.

I wonder who uses this shit. Give me some real markup examples!

Here's the basic template of this web site:

import json

from clabate import MinifiedMarkupTemplate, Markup

class HTMLPageTemplate(MinifiedMarkupTemplate):

    html = Markup('''
        <!DOCTYPE html>
        <html lang="{lang}">
        <head>
            <meta charset="utf-8">
            {meta_viewport}
            {favicon}
            {alternate_langs}
            <title>{page_title}</title>
            {schema_ld_json}
            {social_data}
            {head_stylesheets}
            {head_scripts}
        </head>
        <body{body_attributes}>
            {header}
            {main}
            {footer}
        </body>
        </html>
    ''')

    lang = '{layout.lang}'

    meta_viewport = Markup('<meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no">')

    favicon = Markup('<link rel="shortcut icon" href="/favicon.ico">')

    main = Markup('''
        <main{main_attributes}>
            {main_content}
        </main>
    ''')

    page_title = 'My Web Site'

    schema_ld_json = ''
    social_data = ''

    head_stylesheets = ''
    head_scripts = ''

    body_attributes = ''
    main_attributes = ''
    main_content = ''

    header = ''
    footer = ''

    @property
    def alternate_langs(self, context):
        return Markup.join('', (
            f'<link rel="alternate" hreflang={self.escape.attr(alt.lang)} href={self.escape.attr(alt.url)} />'
            for alt in context.layout.alternate
        ))

    def ld_json(self, data):
        '''
        Helper method to make ld+json markup.
        '''
        json_data = self.escape(json.dumps(data, sort_keys=True, ensure_ascii=False))
        return Markup(f'<script type="application/ld+json">{json_data}</script>')

Although this class contains some defaults, they are defined for sake of clarity. Actual substitutions are defined in subclasses. If you want to see more, you can dive into this shit declassed.art/file/d90c0c43b72c/desige_ext/templates This is based on Desige, declassed site generator, which is a mix of colors on my palette, not an artwork yet.

Also, you can take a look at markup_examples.py.

I want to know more about properties!

Properties give you total freedom and the way to shoot in the foot. Clabate escapes everything, including strings returned by property getters. However, if you construct a Markup, say, from f-strings, you have to take care of escaping by yourself, calling self.escape or self.escape.attr when necessary. Although, the safest way is to call self.render_str and return its result as is.

If you format a template string in a property getter, you may need to call self.dedent to maintain indentation. Clabate calls this helper for class attributes, but does not know what's inside your custom code.

In property getters you can use already rendered values from the context. However, in such cases Clabate might be unable to deduce the right formatting order. You may need to give a hint using depends_on decorator:

from clabate import depends_on

@property
@depends_on('long_description page_title')
def webpage_schema(self, context):
    return self.ld_json({
        # ...
        "headline": context.page_title,
        "description": context.long_description
    })

Where is the documentation?

You're reading it, and if you have read till this line, you're already an expert in Clabate.

Actually, I was going to write it but then realized I have more important things to do.

Where's a test suite???

True coders do not test their code. Testing is for cowards and for those who lacks self-confidence.

IMHO, all testing frameworks are anti-human so far. And while I'm waiting for something amazing in this area, I have to use a shitty script to collect and run all those *_example functions. You can easily write your own or you can try to contribute to this project and convert all those examples to a test suite.

I dislike Mercurial! How can I get Clabate?

pip install clabate

Also, github is available, but don't expect I'll do anything there.