Add source code syntax highlighting to your Django content with Pygments

iBoris

A nice way to add source code syntax highlighting to your Django content is to use Pygments, a python syntax highlighter. My Django content is generated from HTML, RST and has support for markdown, so here's how I integrated pygments.

The following template tag is working for both html and markdown content. I just replaced str(...) by unicode(...) for unicode support in this Djangosnippet. Unfortunately I couldn't post a comment there even by logging in on DjangoSnippets.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
#! /usr/bin/env python
# -*- coding: utf-8 -*-
from django import template
register = template.Library()

# Pygments: http://pygments.org -- a generic syntax highlighter.
from pygments import highlight
from pygments.formatters import HtmlFormatter
from pygments.lexers import get_lexer_by_name, guess_lexer

# Python Markdown (dropped in my project directory)
from markdown import markdown

# BeautifulSoup: http://www.crummy.com/software/BeautifulSoup/
from BeautifulSoup import BeautifulSoup

@register.filter
def render(content, safe="unsafe"):
    """Render this content for display."""

    # First, pull out all the <code></code> blocks, to keep them away
    # from Markdown (and preserve whitespace).
    soup = BeautifulSoup(unicode(content))
    code_blocks = soup.findAll(u'code')
    for block in code_blocks:
        block.replaceWith(u'<code class="removed"></code>')

    # Run the post through markdown.
    if safe == u"unsafe":
        safe_mode = False
    else:
        safe_mode = True
    markeddown = markdown(unicode(soup), safe_mode=safe_mode)

    # Replace the pulled code blocks with syntax-highlighted versions.
    soup = BeautifulSoup(markeddown)
    empty_code_blocks, index = soup.findAll(u'code', u'removed'), 0
    formatter = HtmlFormatter(cssclass=u'source')
    for block in code_blocks:
        if block.has_key(u'class'):
            # <code class='python'>python code</code>
            language = block[u'class']
        else:
            # <code>plain text, whitespace-preserved</code>
            language = u'text'
        try:
            lexer = get_lexer_by_name(language, stripnl=True, encoding=u'UTF-8')
        except ValueError, e:
            try:
                # Guess a lexer by the contents of the block.
                lexer = guess_lexer(block.renderContents())
            except ValueError, e:
                # Just make it plain text.
                lexer = get_lexer_by_name(u'text', stripnl=True, encoding=u'UTF-8')
        empty_code_blocks[index].replaceWith(
                highlight(block.renderContents(), lexer, formatter))
        index = index + 1

    return unicode(soup)

To include source code in HTML, simply use the code tag, for example:

<code class"python">
    print 'Hello world!'
</code>

This will result in:

print 'Hello world!'

For RST(reStructuredText) content, you need the pygments rst-directive, that should be loaded by your project, for example in one of your project installed apps __init__.py or models.py.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
"""
    The Pygments reStructuredText directive
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

    This fragment is a Docutils_ 0.4 directive that renders source code
    (to HTML only, currently) via Pygments.

    To use it, adjust the options below and copy the code into a module
    that you import on initialization.  The code then automatically
    registers a ``code-block`` directive that you can use instead of
    normal code blocks like this::

        .. code-block:: python

            My code goes here.

    If you want to have different code styles, e.g. one with line numbers
    and one without, add formatters with their names in the VARIANTS dict
    below.  You can invoke them instead of the DEFAULT one by using a
    directive option::

        .. code-block:: python
            :linenos:

            My code goes here.

    Look at the `directive documentation`_ to get all the gory details.

    .. _Docutils: http://docutils.sf.net/
    .. _directive documentation:
       http://docutils.sourceforge.net/docs/howto/rst-directives.html

    :copyright: 2007 by Georg Brandl.
    :license: BSD, see LICENSE for more details.
"""

# Options
# ~~~~~~~

# Set to True if you want inline CSS styles instead of classes
INLINESTYLES = False

from pygments.formatters import HtmlFormatter

# The default formatter
DEFAULT = HtmlFormatter(noclasses=INLINESTYLES)

# Add name -> formatter pairs for every variant you want to use
VARIANTS = {
    'linenos': HtmlFormatter(noclasses=INLINESTYLES, linenos=True),
}


from docutils import nodes
from docutils.parsers.rst import directives

from pygments import highlight
from pygments.lexers import get_lexer_by_name, TextLexer

def pygments_directive(name, arguments, options, content, lineno,
                       content_offset, block_text, state, state_machine):
    try:
        lexer = get_lexer_by_name(arguments[0])
    except ValueError:
        # no lexer found - use the text one instead of an exception
        lexer = TextLexer()
    # take an arbitrary option if more than one is given
    formatter = options and VARIANTS[options.keys()[0]] or DEFAULT
    parsed = highlight(u'\n'.join(content), lexer, formatter)
    parsed = '<div class="codeblock">%s</div>' % parsed
    return [nodes.raw('', parsed, format='html')]

pygments_directive.arguments = (1, 0, 1)
pygments_directive.content = 1
pygments_directive.options = dict([(key, directives.flag) for key in VARIANTS])

directives.register_directive('code-block', pygments_directive)

More on this integration can be found on Code Koala.

To include source code in RST, simply use the code-block directive, for example:

.. code-block:: python

  print 'Hello world!'

This will result in:

print 'Hello world!'

You will also need the following installed: pygments, docutils (for RST support), beautifulsoup and markddown.

$ sudo apt-get install python-setuptools
$ sudo easy_install Pygments
$ sudo easy_install Beautifulsoup
$ sudo easy_install markdown
$ sudo easy_install docutils

Finally, all you need is to create or generate a css style sheet and add it to your templates/pages. Check http://pygments.org/demo/ for code and styling examples

List all the builtin pygments styles:

>>> from pygments.styles import STYLE_MAP
>>> STYLE_MAP.keys()
['manni', 'perldoc', 'borland', 'colorful', 'default', 'murphy', 'trac', 'fruity', 'autumn', 'emacs', 'pastie',
'friendly', 'native']

And generate builtin style css:

$ pygmentize -S colorful -f html > pygments-colorful.css

Links:

  • Djangosnippet - Markdown and Syntax Highlighting in Django
  • Code Koala - Syntax Highlighting, ReST, Pygments, and Django

Django

Technology latest entries

London during the Christmas season

Who is Santa Grotto ?

Who is Santa Grotto ?