Interpreters

How to build an interpreter

Building an interpreter comes down to five stages: scan the source into tokens, parse the tokens into a tree, resolve what each name refers to, then either evaluate the tree directly or compile it to bytecode and run that on a virtual machine. The best way to learn it is to build one complete, small language end to end, and the best guide for that is Robert Nystrom's free book Crafting Interpreters.

Updated

The stages

  1. Scanning (lexing). Turn print 1 + 2; into tokens: PRINT, NUMBER 1, PLUS, NUMBER 2, SEMICOLON. A loop over characters with a switch.
  2. Parsing. Turn tokens into an abstract syntax tree that captures structure and precedence. A hand-written recursive descent parser is the usual choice.
  3. Resolving. Work out, before running, which declaration each variable use refers to. This is what makes closures correct.
  4. Evaluating. Walk the tree and compute values: a tree-walk interpreter.
  5. Or compiling. Emit bytecode and run it on a VM, with your own memory management and garbage collector. Much faster, much more work.

Pick a small language and finish it

The classic mistake is designing a big language and stalling on the parser. Start with a language small enough to finish: numbers, strings, variables, if, while, functions, closures, and maybe classes. Lox, the language in Crafting Interpreters, is exactly that, and it comes with a test suite of about 250 programs with expected output, so you always know whether your implementation is right. See running the Lox test suite.

Build it twice

Crafting Interpreters builds Lox twice. First jlox, a tree-walker in Java, where the JVM handles memory and you focus on the language. Then clox, a bytecode VM in C, where you build the hash table, the closures and the garbage collector yourself. Doing both teaches you why real languages are built the way they are. See jlox vs clox.

How long it takes

The book has 30 chapters. Typing the code yourself and doing some of the challenges, plan on about 160 hours: the cracked.guide reading guides budget 57 hours for the jlox chapters and 100 for clox. Full time that is about a month; the cracked.guide path gives it 28 weeks, a chapter a week next to two NeetCode problems, so a week stays around 8 to 10 hours. See the Crafting Interpreters study plan or generate dated weeks.

Tips that save weeks

  • Type the code, do not paste it. The typing is where you learn it.
  • Run the tests every chapter. Push your repo and let GitHub Actions run the official suite.
  • Do challenges on a branch so a language change does not break the tests.
  • Compile C with sanitizers (-fsanitize=address,undefined) from the first C chapter.
  • Draw it. Environments, the value stack, upvalues: draw them for small examples before writing code.

Questions

What is the best book for building an interpreter?

Crafting Interpreters by Robert Nystrom is the usual recommendation: free online, complete, and it builds a real language twice with tests for every chapter.

Interpreter or compiler first?

An interpreter. You get a working language much sooner, and a bytecode VM teaches most of what a compiler back end does.

What language should I write my interpreter in?

The book uses Java and C. Any language works if you are willing to translate; Rust, Go, Python and TypeScript ports are common.