Reading remote files

Won’t it be nice if TeX could pretty-print files hosted on github, e.g.,

\typeRUBYfile{https://raw.github.com/adityam/filter/master/Rakefile}

or include a remotely hosted markdown file in your document

\processmarkdownfile{https://raw.github.com/adityam/filter/master/README.md}

I wanted to add this feature to the filter and vim modules.
Although I knew that ConTeXt could read remote files directly, I thought that it would be hard to plug into this mechanism.

Boy, was I wrong. Look at the commit history of the change needed to add this feature.

All I needed to do, was add \locfilename to get the local file name for a file. If the requested file is a remote file (i.e., starts with http:// or ftp://), ConTeXt downloads the file and stores it in the cache directory, and return the name of the cached file. Pretty neat, eh?

With this change, \process<filter>file macro of the filter module can read remote files. Since, the vim module is built on top of the filter module, the \type<vim>file can also read remote files.

The above feature is currently only available in the dev branch. I’ll make a new release once I add hooks to force re-download of remote files. Meanwhile, if you have a ConTeXt macro that reads files, just add a \locfilename at appropriate place, and your macro will be able to read remote files

Update for the filter module: faster caching

Over the last year, the code base of the filter module has matured considerably. Now, the module has all the features that I wanted when I started with it about a year and a half back. The last remaining limitation (in my eyes, at least) was that caching of results required a call to external programs (mtxrun) to calculate md5 hashes; as such, caching was slow. That is no longer the case. Now (since early December), md5 sums are calculated at the lua end, so there is no time penalty for caching. As a result, in MkIV, recompiling is much faster for documents having lots of external filter environments with caching enabled(i.e., environments defined with continue=yes option).

Some thoughts on lowering the learning curve for using TeX (part I)

TeX has a steep learning curve. Often times, steeper than it needs to be. Take, for example, the special characters in TeX. Almost every introduction to plain TeX, eplain, LaTeX, or ConTeXt has a section on these special characters

\ { } $ & # ^ _ & ~

A good introduction then goes on to explain why these special characters are important; sometimes dropping a hint about category codes. I feel that these details are useless and, at the user level, we should get rid of them.

Typewriter scroll mode in vim

I came across a Mac specific application Writeroom for distraction free writing and couldn’t help wonder that good ‘ol vim provides the same features.

One interesting feature was "typewriter scroll mode", in which, the line that you are writing is always in the middle of the screen. At times, while editing long files, I find myself too often at the bottom of the screen at find myself typing zz to redraw the buffer so that the current line is at the middle of the screen. It would be nice to have that as default. A quick search of the vim help revealed:

'scrolloff' 'so'    number  (default 0)
            global
            {not in Vi}
    Minimal number of screen lines to keep above and below the cursor.
    This will make some context visible around where you are working.  If
    you set it to a very large value (999) the cursor line will always be
    in the middle of the window (except at the start or end of the file or
    when long lines wrap).
    For scrolling horizontally see 'sidescrolloff'.
    NOTE: This option is set to 0 when 'compatible' is set.

So, the solution is:

set scrolloff=99

visual counter for Maya numbers

The other day I saw the documentary breaking the Maya code. At one stage they showed how the Maya number system worked. If you haven’t seen the documentary, see the wikipedia page on Maya number system. I thought that this would be a nice way to visually represent page numbers in presentations. So, I added a new style to the visual counter, unimaginatively called mayanumbers. A test file comparing this affect with other visual counters from the module is in the github directory of the module.

Have a look at the output pdf. It uses four counters; cycically from the top-left corner they are: countdown, pulseline, scratchcounter, and finally, mayanumbers. !’ll definitely experiment with mayanumbers in my next presentation.

Syntax highlighting engines: clean tex output

The vim module uses the vim editor to syntax highlight code snippets in ConTeXt. I thought that it should be straight forward to support other syntax highlighting engines: source-highlight, pygments, HsColor, etc. Unfortunately, that is not the case. None of these syntax highlighting engines were written with reuse in mind.

For example, consider a simple tex file:

\definestartstop[important]
                [color=red,
                 style=\italic]
\starttext
This is an \important{important} text
\stoptext

Lets compare the tex file generated by various syntax highlighters:

source-highlight -f latex gives

% Generator: GNU source-highlight, by Lorenzo Bettini, http://www.gnu.org/software/src-highlite
\noindent
\mbox{}\textbf{\textcolor{Blue}{\textbackslash{}definestartstop}}\textcolor{Purple}{[important]} \\

\mbox{}\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ [color=red, \\

\mbox{}\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ style=\textbf{\textcolor{Blue}{\textbackslash{}italic}}] \\

\mbox{}\textbf{\textcolor{Blue}{\textbackslash{}starttext}} \\

\mbox{}This\ is\ an\ \textbf{\textcolor{Blue}{\textbackslash{}important}}\textcolor{ForestGreen}{\{important\}}\ text \\

\mbox{}\textbf{\textcolor{Blue}{\textbackslash{}stoptext}} \\

\mbox{}

pygmentize -f latex gives

\begin{Verbatim}[commandchars=\\
\{\}]
\PY{k}{\PYZbs{}definestartstop}\PY{n+na}{[important]}
                [color=red,
                 style=\PY{k}{\PYZbs{}italic}]
\PY{k}{\PYZbs{}starttext}
This is an \PY{k}{\PYZbs{}important}\PY{n+nb}{\PYZob{}}important\PY{n+nb}{\PYZcb{}} text
\PY{k}{\PYZbs{}stoptext}
\end{Verbatim}

HsColor-latex -partial gives

\textcolor{red}{$\backslash$}{\rm{}definestartstop}\textcolor{red}{[}{\rm{}important}\textcolor{red}{]}\\
\hsspace \hsspace \hsspace \hsspace \hsspace \hsspace \hsspace \hsspace \hsspace \hsspace \hsspace \hsspace \hsspace \hsspace \hsspace \hsspace \textcolor{red}{[}{\rm{}color}\textcolor{red}{=}{\rm{}red}\textcolor{cyan}{,}\\
\hsspace \hsspace \hsspace \hsspace \hsspace \hsspace \hsspace \hsspace \hsspace \hsspace \hsspace \hsspace \hsspace \hsspace \hsspace \hsspace \hsspace {\rm{}style}\textcolor{cyan}{=$\backslash$}{\rm{}italic}\textcolor{red}{]}\\
\textcolor{red}{$\backslash$}{\rm{}starttext}\\
{\rm{}This}\hsspace {\rm{}is}\hsspace {\rm{}an}\hsspace \textcolor{red}{$\backslash$}{\rm{}important}\textcolor{cyan}{\{}{\rm{}important}\textcolor{cyan}{\}}\hsspace {\rm{}text}\\
\textcolor{red}{$\backslash$}{\rm{}stoptext}\\

HsColor and source-highlight use explicit LaTeX commands for spacing and formatting. Ouch! Pygments uses logical markup, but with cryptic command names. But, from the point of view of using pygments output in ConTeXt, the \begin{Verbatim} and \end{Verbatim} are show stopper. (OK, not really. It can be bypassed with some effort).

Based on my experience, I decided to clean up the output generated by 2context.vim:

\SYN[Identifier]{\\definestartstop}[important]
                [color=red,
                 style=\SYN[Identifier]{\\italic}]
\SYN[Statement]{\\starttext}
This is an \SYN[Identifier]{\\important}\{important\} text
\SYN[Statement]{\\stoptext}

I assume only four TeX commands to be defined: \\, \{, and \} for backslash, open brace, and close brace; and \SYN[...]{...} for syntax highlighting. Thus, if anyone wants to reuse 2context in plain TeX or LaTeX, or a yet to be written future macro package, they would not need to modify the output at all. I wish the other syntax highlighting programs did the same.

HTML export

The question of translating TeX to (X)HTML arises frequently. Almost everyone wants it. After all, on the web, (X)HTML is the de-facto standard markup; PDF, with all its hyper link abilities, is clumsier to use. On the other hand, for print, PDF, especially, TeX generated PDF is the de-facto standard (at least for math heavy fields); (X)HTML, with all its print css abilities, is clumsier to use. Often, you want both an (X)HTML version and a PDF version of a document. With the popularity of eink devices, epub (which is essentially a zipped (X)HTML file) is also becoming popular. Generating these multiple output formats from the same source is tricky.