Beating an old horse back to life — the t-vim module

When including a source code listing in a document, I want (colored) syntax highlighting for the code. For HTML output, a huge number of tools are available for the last. However, most of the time, I want to generate PDF output using ConTeXt. ConTeXt does support syntax highlighting for about a dozen languages, but it does not support the languages that  I use regularly — MATLAB and R. Supporting syntax highlighting for a new language is not trivial—I had a headache the last time I tried! This is where the t-vim module comes in.

The idea of the module, like a lot of ideas in ConTeXt mailing list, was germinated  by Mojca Miklavec. She said (crica Sep 2005):

I am thinking of piping the code to vim, letting vim process it, and return something like

\highlight[Conditional]{if} \highlight[Delimiter]{(} \highlight[Identifier]{!}

One could modify the 2html.vim file. Vim can already transform the highlighted code to HTML, so ConTeXt should not be so difficult. Vim already has over 400 syntax file definitions, probably equivalent to some hundred thousand lines of syntax definition in ConTexT. Well, I don’t know (yet) how to do it, but if someone on the last has more experience with vim, please feel free to contribute.

A few months later (circa Dec 2005), Nikolai Webull provided such a modification of 2html.vim and called it 2context.vim. That file was the foundation of t-vim module.

About two years later, Mojca and I pickup up on this idea and released t-vim. In the last year years, neither I nor Mojca have done any maintenance of the module, and it has started showing signs of bit rot. Every four or five months, we get a bug report that says that the module does not work. I have finally decided to revive this module.

As a first step, I have cleaned up the 2context script. The generated ConTeXt output is now much cleaner than before. For example, the following C file


void main()
    printf("Hello world & its people \n") ;

gets converted to

\NL{}\SYN[Type]{void}\SYN[]{ main()}
\NL{}\SYN[]{    printf(}\SYN[Constant]{"Hello world & its people }\SYN[Special]{\letterbackslash{}n}\SYN[Constant]{"}\SYN[]{) ;}

As you can see, each line starts with \NL (new line). The generated output is logical and only \, {, } are escaped. To use this output in ConTeXt, all I need to do is provide a ConTeXt macro


that generates the correct highlighting. Providing an environment that writes its contents to an external file, runs vim on it, and reads the output may be delegated to the t-filter module (that I blogged about earlier). A proof of concept reimplementation of the t-vim module is here, but I want to write a module that will easily allow us to use any external program (like pgyments) for syntax highlighting. Once that is done, t-vim can retire respectfully.