Question

I’m doing some small projects which involve having different syntaxes for something, however sometimes these syntaxes are so easy that using a parser generator might be overkill.

Now, when should I use a hand-made parser, and when should I use a parser generator?

Thanks,

William van Doorn

Answer 1

It’s worth coding the parser by hand if, and only if, you’re super-keen to have it be extremely fast even on a machine of very modest speed. For example, in this article on the history of Turbo Pascal from before it got its name, you can see how and why the prototype impressed the small (then Danish) firm “Borland” to hire the prototype’s author (Anders Hejlsberg), fully develop the compiler, and launch it as its main product, and I quote…:

with no great expectations I hit the
compile key – AND THEN I WAS
COMPLETELY FLOORED! My test program,
that took minutes to compile and link
using Digital Research’s Pascal MT+,
was compiled and running before I
could blink an eye! That was a great
WOW moment!

Turbo Pascal’s amazing compile speed — coming first and foremost from a carefully hand-coded and highly tuned recursive descent parser coded in assembly language — allowed it to use a very different strategy from most compilers: no separate compilation pass generating object files and libraries, and then a linker to put them together, rather, Turbo Pascal 1.0 was a single-pass compiler that directly turned source code into a single executable binary.

I remember just the same amazing experience on the tiny personal computers of that era (when a Z80, 64K or RAM, and two floppies was a lot😉 — Turbo Pascal, with its amazing parser and the IDE and everything else, fit comfortably in memory together with a substantial program in both source and compiled form — no floppies were needed, which meant many orders of magnitude of difference in program turnaround time.

If Hejlsberg had stuck to what was already the traditional wisdom at the time — always use parser generators — Turbo Pascal would probably never have emerged as a commercial product, and definitely not achieved the dominance in the Pascal world it enjoyed for years.

Of course, on a typical PC of today, such extreme parsing speed would not be needed for most compilers. Possible exceptions include compilers that must run seamlessly as part of an “interpreter-like” environment (the simple compilers for languages such as Perl and Python are typically hand-coded, to substantial extents, for that reason — that was an implementation choice that made them viable in the ’90s, although today it’s not clear it’s still needed), or compilers that run on very limited hardware resources, such as smartphones or low-end netbooks.

In the vast majority of cases in which you’ll be writing a compiler, none of these performance considerations probably apply, and you’ll be happier with a parser generator.