A high-throughput parser for the Zig programming language

This is very cool. Extremely fast lexical tokenizer is the basis for a fast compiler. Zig has good integration and support for SIMD operations that&#x27;s perfect for this kind of things. It&#x27;s definitely doable. I did a proof of concept on using SIMD to operate on 32-byte chunk to parse identifiers a while back.<p><a href="https:&#x2F;&#x2F;github.com&#x2F;williamw520&#x2F;misc_zig&#x2F;blob&#x2F;main&#x2F;identifier.zig">https:&#x2F;&#x2F;github.com&#x2F;williamw520&#x2F;misc_zig&#x2F;blob&#x2F;main&#x2F;identifier...</a>

The talks that Niles gave at the Utah Zig meetups (linked in the repo) were great, just wished the AV setup was a little smoother. There seemed like there some really neat visualizations that Niles prepared that flopped. Either way, I recommend it. Inspired me to read a lot more machine code these days.

Very interesting project!<p>I wonder if there&#x27;s a way to make this set of techniques less brittle and more applicable to any language. I guess you&#x27;re looking at a new backend or some enhancements to one of the parser generator tools.

Would be very cool, if once finished, the techniques are applied to user-schedulable languages <a href="https:&#x2F;&#x2F;www.hytradboi.com&#x2F;2025&#x2F;7d2e91c8-aced-415d-b993-f6f85f6426c1-safe-and-productive-performance-with-user-schedulable-languages" rel="nofollow">https:&#x2F;&#x2F;www.hytradboi.com&#x2F;2025&#x2F;7d2e91c8-aced-415d-b993-f6f85...</a>.<p>I guess they are too tailored to the actual memory layout with respective memory access delay of the architecture, but I would like to be shown that I am wrong and it is feasible.

This really moves Zig