TE
TechEcho
Home24h TopNewestBestAskShowJobs
GitHubTwitter
Home

TechEcho

A tech news platform built with Next.js, providing global tech news and discussions.

GitHubTwitter

Home

HomeNewestBestAskShowJobs

Resources

HackerNews APIOriginal HackerNewsNext.js

© 2025 TechEcho. All rights reserved.

My Most Important Project Was a Bytecode Interpreter

313 pointsby 10098over 8 years ago

22 comments

robertelderover 8 years ago
One of the moments where I really started to feel like I was starting to &#x27;see the matrix&#x27; was when I was working on a regex engine to try to make my compiler faster (it didn&#x27;t, but that&#x27;s another story). The asymptotically fast way to approach regex processing actually involves writing a parser to process the regex, so in order to write a fast compiler, you need to write another fast compiler to process the regexes that will process the actual programs that you write. But, if your regexes get complex, you should really write a parser to parse the regexes that will parse the actual program. This is where you realize that it&#x27;s parsers all the way down.<p>When you think more about regexes this way, you realize that a regex is just a tiny description of a virtual machine (or emulator) that can process the simplest of instructions (check for &#x27;a&#x27;, accept &#x27;0-9&#x27;, etc.). Each step in the regex is just a piece of bytecode that can execute, and if you turn a regex on its side you can visualize it as just a simple assembly program.
评论 #12554201 未加载
评论 #12554832 未加载
评论 #12557351 未加载
评论 #12553996 未加载
评论 #12557417 未加载
sillysaurus3over 8 years ago
Also: a software rasterizer.<p>Most people refuse to write one because it&#x27;s so easy not to. Why bother?<p>It will make you a better coder for the rest of your life.<p>Let&#x27;s make a list of &quot;power projects&quot; like this. A bytecode interpreter, a software rasterizer... What else?
评论 #12554033 未加载
评论 #12554115 未加载
评论 #12553913 未加载
评论 #12554969 未加载
评论 #12553931 未加载
评论 #12553896 未加载
评论 #12554016 未加载
评论 #12554486 未加载
评论 #12555453 未加载
评论 #12556860 未加载
评论 #12553997 未加载
评论 #12555258 未加载
评论 #12554048 未加载
评论 #12553958 未加载
评论 #12554057 未加载
评论 #12554044 未加载
评论 #12560185 未加载
评论 #12565200 未加载
评论 #12555838 未加载
评论 #12560186 未加载
tominousover 8 years ago
I love the author&#x27;s meta-idea of refusing to accept that unfamiliar things are black boxes full of magic that can&#x27;t be touched.<p>A great example of this mindset is the guy who bought a mainframe. [1]<p>Refuse to be placed in a silo. Work your way up and down the stack and you&#x27;ll be much better placed to solve problems and learn from the patterns that repeat at all levels.<p>[1] <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=11376711" rel="nofollow">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=11376711</a>
评论 #12554131 未加载
wahernover 8 years ago
Two approaches are severely underused in the software world:<p>1) Domain-specific languages (DSLs)<p>2) Virtual machines (or just explicit state machines more generally)<p>What I mean is, alot of problems could be solved cleanly, elegantly, more safely, and more powerfully by using one (or both) of the above. The problem is that when people think DSL or VM, they think big (Scheme or JVM) instead of thinking small (printf). A DSL or VM doesn&#x27;t need to be complex; it could be incredibly simple but still be immensely more powerful than coding a solution directly in an existing language using its constructs and APIs.<p>Case in point: the BSD hexdump(1) utility. POSIX defines the od(1) utility for formatting binary data as text, and it takes a long list of complex command-line arguments. The hexdump utility, by contrast, uses a simple DSL to specify how to format output. hexdump can implement almost every conceivable output format of od and then some using its DSL. The DSL is basically printf format specifiers combined with looping declarations.<p>I got bored one day and decided to implement hexdump as a library (i.e. &quot;one hexdump to rule them all&quot;), with a thin command-line wrapper that emulates the BSD utility version. Unlike BSD hexdump(1) or POSIX od(1), which implement everything in C in the typical manner, I decided to translate the hexdump DSL into bytecode for a simple virtual machine.<p><pre><code> http:&#x2F;&#x2F;25thandclement.com&#x2F;~william&#x2F;projects&#x2F;hexdump.c.html </code></pre> The end result was that my implementation was about the same size as either of those, but 1) could built as a shared library, command-line utility, or Lua module, 2) is more performant (formats almost 30% faster for the common outputs, thanks to a couple of obvious, easy, single-line optimizations the approach opened up) than either of the others, and 3) is arguably easier to read and hack on.<p>Granted, my little hexdump utility doesn&#x27;t have much value. I still tend to rewrite a simple dumper in a couple dozen lines of code for different projects (I&#x27;m big on avoiding dependencies), and not many other people use it. But I really liked the experience and the end result. I&#x27;ve used simple DSLs, VMs, and especially explicit state machines many times before and after, but this one was one of the largest and most satisfying.<p>The only more complex VM I&#x27;ve written was for an asynchronous I&#x2F;O SPF C library, but that one is more difficult to explain and justify, though I will if pressed.
评论 #12555578 未加载
评论 #12555942 未加载
评论 #12554140 未加载
gopalvover 8 years ago
The project that affected my thinking the most was a bytecode interpreter[1].<p>I&#x27;ve had use for that knowledge, nearly fifteen years later - most of the interesting learnings about building one has been about the inner loop.<p>The way you build a good interpreter is upside-down in tech - the system which is simpler often works faster than anything more complicated.<p>Because of working on that, then writing my final paper about the JVM, contributing to Perl6&#x2F;Parrot and then moving onto working on the PHP bytecode with APC, my career went down a particular funnel (still with the JVM now, but a logical level above it).<p>Building interpreters makes you an under-techtitect, if that&#x27;s a word. It creates systems from the inner loop outwards rather than leaving the innards of the system for someone else to build - it produces a sort of double-vision between the details and the actual goals of the user.<p>[1] - &quot;Design of the Portable.net interpreter&quot;
评论 #12554308 未加载
_RPMover 8 years ago
I saw the matrix after I first implemented a virtual machine. I recommend everyone does it because it will teach you a lot about how code is executed and transformed from the syntax to the actual assembly&#x2F;bytecode. A stack based virtual machine is so simple it takes a lot of thinking to understand how they work. (or maybe I&#x27;m just not that smart).<p>It&#x27;s interesting that he implemented function calls via a jump. In my VM a function is just mapped to a name (variable), so functions are first class. When the VM gets to a CALL instruction, it loads the bytecode from the hash table (via a lookup of the name).<p>Since this is a procedural language where statements can be executed outside of a function, implementing the functions as a jump would be difficult because there would need to be multiple jumps between the function definition and statements that aren&#x27;t in a function.<p>I really wish my CS program had a compilers class, but unfortunately they don&#x27;t, so I had to learn everything on my own.
评论 #12555896 未加载
briansteffensover 8 years ago
Nice post! I really enjoy playing around with things like this. It&#x27;s amazing how little is needed to make a language&#x2F;interpreter capable of doing virtually anything, even if not elegantly or safely. As long as you can perform calculations, jump around, and implement some kind of stack your language can do just about anything.<p>I recently threw something together sort of like this, just for fun (I like your interpreter&#x27;s name better though): <a href="https:&#x2F;&#x2F;github.com&#x2F;briansteffens&#x2F;bemu" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;briansteffens&#x2F;bemu</a><p>It&#x27;s crazy how much these little projects can clarify your understanding of concepts that seem more complicated or magical than they really are.
anaccountwowover 8 years ago
This is a required hw assignment for a freshmen class @ cmu. <a href="https:&#x2F;&#x2F;www.cs.cmu.edu&#x2F;~fp&#x2F;courses&#x2F;15122-s11&#x2F;lectures&#x2F;23-c0vm.pdf" rel="nofollow">https:&#x2F;&#x2F;www.cs.cmu.edu&#x2F;~fp&#x2F;courses&#x2F;15122-s11&#x2F;lectures&#x2F;23-c0v...</a><p>Given it has some parts already written in the interest of time...
评论 #12556431 未加载
评论 #12554483 未加载
oopsover 8 years ago
Nice read! Reminds me of nand2tetris that was posted not too long ago <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=12333508" rel="nofollow">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=12333508</a><p>(You basically implement every layer starting with the CPU and finishing with a working Tetris game)
doucheover 8 years ago
This reminds me a little bit of my computer architecture class. We started at logic gates in a simulator[1], and worked our way up from there to flip-flops and adders, memory chips, a simple ALU, and eventually a whole 8-bit CPU in the simulator. I want to think that we were even writing assembly for it, loading the programs into the simulated memory, and executing it. It was a great way to get a sense of how everything works, and I think it&#x27;s when C-style pointers really clicked for me.<p>[1] this one, IIRC <a href="https:&#x2F;&#x2F;sourceforge.net&#x2F;projects&#x2F;circuit&#x2F;" rel="nofollow">https:&#x2F;&#x2F;sourceforge.net&#x2F;projects&#x2F;circuit&#x2F;</a>
foobargeover 8 years ago
I&#x27;ve done something similar 21 years ago: a C interpreter targeting a virtual machine. The runtime had a dynamic equivalent of libffi to call into native code and use existing native libraries. I added extensions to run code blocks in threads so that the dinning philosopher problem solution was very elegant. Back in the days, not having libffi meant generating assembly on the fly for Sparc, MIPS, PA-Risc, i386. Fun times. That C interpreter was used to extend a CAD package.
reidracover 8 years ago
I wrote a VM for the 6502 for fun and it was one of most interesting and satisfying projects I&#x27;ve ever made in my free time.<p>It is very close to a bytecode interpreter, only that it comes with a specification that is actually the opcode list for the MOS 6502 (and few details you need to take into account when implementing that CPU).<p>Besides there are cross-compilers that allows you to generate 6502 code from C for your specific VM (see cc65).
memsomover 8 years ago
I did this in C#. It was a lunch time project at work a couple of years ago. It was fun. I still want to do a V2 and remove all of the shortcuts I put in because I didn&#x27;t want to write code for the stack and stuff like that. At the end of the day, my solution was spookily similar to this - the 32bit instructions - well, yeah, I was the same! It was just simpler. I did have a few general purpose registers (V1, V2 and V3 I think) and I did have routines to handle bytes, words and such like. So stuff like this (as a random example I pulled from the source):<p>ORG START<p>START: ST_B 10<p>LOOP: ST_B 10<p>ADD_B ;;value will go back on stack<p>LD_B V1<p>SM_B V1 ;;value we use next loop<p>SM_B V1 ;;value we compare<p>SM_B V1 ;;value we echo to console<p>TRP 21 ;;writes to the console<p>ST_S &#x27;&#x27;,13,10,$<p>TRP 21 ;;writes to the console<p>CMP_B 50 ;;compares stack to the constant<p>JNE LOOP<p>ST_S &#x27;The End&#x27;,13,10,$<p>TRP 21 ;;writes to the console<p>END
pkaover 8 years ago
I&#x27;m thinking a lot of the complexity of writing a compiler stems from the usage of inappropriate tools. I.e. I would rather kill myself than write a lexer in C (without yacc &#x2F; bison), but using parser combinators it&#x27;s a rather trivial task.<p>Similarly, annotating, transforming, folding, pattern matching on, CPS transforming etc. the produced AST is pretty trivial in a language that supports these constructs. And again, a nightmare in C.<p>That leaves codegen, but using the right abstractions it turns into a very manageable task as well.<p>Here&#x27;s a compiler written in Haskell for LLVM [0].<p>[0] <a href="http:&#x2F;&#x2F;www.stephendiehl.com&#x2F;llvm" rel="nofollow">http:&#x2F;&#x2F;www.stephendiehl.com&#x2F;llvm</a>
评论 #12558996 未加载
philippebackover 8 years ago
Parsers made easy and pretty much interactive:<p><a href="http:&#x2F;&#x2F;www.lukas-renggli.ch&#x2F;blog&#x2F;petitparser-1" rel="nofollow">http:&#x2F;&#x2F;www.lukas-renggli.ch&#x2F;blog&#x2F;petitparser-1</a><p><a href="http:&#x2F;&#x2F;www.themoosebook.org&#x2F;book&#x2F;internals&#x2F;petit-parser" rel="nofollow">http:&#x2F;&#x2F;www.themoosebook.org&#x2F;book&#x2F;internals&#x2F;petit-parser</a><p>This include the dynamic generation of blocks and arrows style things...
philippebackover 8 years ago
Soulmate of yours here: <a href="https:&#x2F;&#x2F;clementbera.wordpress.com" rel="nofollow">https:&#x2F;&#x2F;clementbera.wordpress.com</a><p>Lots of optimizations going on for OpenVM.<p><a href="https:&#x2F;&#x2F;github.com&#x2F;OpenSmalltalk&#x2F;opensmalltalk-vm" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;OpenSmalltalk&#x2F;opensmalltalk-vm</a><p>Interesting bit: VM is written in Slang and transformed into C then compiled.<p>So you can livecode your VM. In the VM simulator.
elcctover 8 years ago
I did something similar in the distant past, that is I wrote subset of C compiler (functions, standard types, pointers) to imaginary assembler and then bytecode interpreter. It was awesome fun, but also I got so into it my - then - girlfriend started to question my commitment to the relationship. So be careful, this is really interesting thing to do :)
rosstexover 8 years ago
In this same vein, I recommend coding an emulator! It can be an excellent experience.<p><a href="http:&#x2F;&#x2F;www.multigesture.net&#x2F;articles&#x2F;how-to-write-an-emulator-chip-8-interpreter&#x2F;" rel="nofollow">http:&#x2F;&#x2F;www.multigesture.net&#x2F;articles&#x2F;how-to-write-an-emulato...</a>
curtfooover 8 years ago
Yes I wrote a parser&#x2F;compiler and interpreter for a custom domain specific language and it had a similar effect on my career. Lots of fun!<p>Okay I guess technically I used a parser generator that I then modified to build an AST and convert it into assembly-like code that fed the interpreter.
reacwebover 8 years ago
Bill gates also started with an interpreter (basic interpreter). Many parts of early windows applications were developed in p-code and visual basic is an important part of Microsoft success.
loegover 8 years ago
I like implementing emulators, because the toolchain and architecture specification are all there already. You get to implement what is basically a little embedded CPU.
dprattover 8 years ago
I&#x27;d add a driver for a non trivial binary protocol - I ended up implementing a JVM driver for Cassandra a few years ago, and it was a blast.
评论 #12553985 未加载