Crafting Interpreters · Week 9
Crafting Interpreters 16: Scanning on Demand: reading guide
A new scanner for clox that produces tokens on demand instead of all at once, pointing into the source string rather than copying lexemes, with a hand-written trie-like switch for keywords.
Time About 4 hours this week
Why it matters
Zero-copy scanning is how fast real-world parsers work. The keyword switch is a neat trick worth stealing.
How to approach it
Compare each function with your jlox scanner. Note how lexemes are pointers plus lengths, not strings.
Challenges
3 challenges at the end of the chapter.
Challenge 1 (string interpolation) is a great design problem; think through how the scanner would need to cooperate with the parser. Challenges 2 and 3 are design discussions.
Traps
- Token lexemes are not null-terminated. Never pass them to printf with %s; use %.*s.
- The source buffer must outlive all tokens.