I just published a bidirectional code generation library. Afaik it's the first of its kind, and it opens up a lot of possibilities for cool new types of dev tools. The PoC is for ruby, but the concept is very portable. <a href="https://blog.luitjes.it/posts/monocle-bidirectional-code-generation/" rel="nofollow">https://blog.luitjes.it/posts/monocle-bidirectional-code-gen...</a>
There is a fair amount of academic work on bidirectional tree transformation with lenses, e.g. <<a href="https://www.cs.cornell.edu/~jnfoster/papers/lenses.pdf" rel="nofollow">https://www.cs.cornell.edu/~jnfoster/papers/lenses.pdf</a>>. It breaks down to proving three operations (Get, Put, Create) that observe three laws (called GetPut, PutGet, and CreateGet); these give you bidirectional transformations you can compose arbitrarily. Later work introduces concepts like "quotienting" (for when you actually want the transformation to be lossy in certain ways) or "discerning" (non-total) lenses.
Super cool. Bidirectional code generation is something I've spent a bit of time thinking about: I've been building a spreadsheet that generates Python code when you edit it [1], but some of our users also want the ability to edit the Python code they generate and have that reflect in the sheet itself.<p>Template -> Code -> Template is one really hard part of this, and something this tool seems to take a really good approach to. If you're interested in related subjects (like transpiliation), I'd recommend this overview article as a great approach [2]. From this article: "Now, there is a single biggest mistake we see in persons trying to implement a transpiler without experience in this: they try to generate directly the code of the target language from the AST of the original language."<p>Question to the OP - you mention you parse the Ruby AST - do you also transform this into an AST of your template syntax before generating the template? Aka, do you avoid this sin, or is it not an issue for you?<p>With Mito, there is additional complexity beyond just going from Template -> Code -> Template, in that we also need to understand _which_ variables are being changed and in what way. This is necessary because a spreadsheet stores other data about your variables beyond just their current value. As an example, which of these columns in a dataframe are a result of a formula vs. being in the original dataset isn't something that is just stored in the dataframe itself.<p>I haven't tried too hard, but I don't think there's a general solution; it feels like it requires some sort of symbolic execution in the general case, and is tough to do well even in simple cases. Our "fail loudly and early" equivalent feels like it would be a lot higher than the 10-20% this tool can deliver!<p>Anyways, bidirectional spreadsehet code generation is low on the priorities... but it's a fun one to dream about :-)<p>[1] <a href="https://trymito.io" rel="nofollow">https://trymito.io</a>
[2] <a href="https://tomassetti.me/how-to-write-a-transpiler/" rel="nofollow">https://tomassetti.me/how-to-write-a-transpiler/</a>
Whoops, looks like I should've posted this as a link rather than text containing a link. Here's something clickable: <a href="https://blog.luitjes.it/posts/monocle-bidirectional-code-generation/" rel="nofollow">https://blog.luitjes.it/posts/monocle-bidirectional-code-gen...</a>
Yeah back in the 90's it was called "round-tripping"<p><a href="https://www.ibm.com/docs/en/rhapsody/8.2?topic=developing-roundtripping-code" rel="nofollow">https://www.ibm.com/docs/en/rhapsody/8.2?topic=developing-ro...</a><p>I did a lot of code generation work in those years, working on the two dominant Mac-based generators (AppMaker and Prototyper) but was never ambitious enough to try round-tripping because of the horrors of parsing C++.
This is a powerful idea! If I understand Monocle's use case, JS has similar AST parsing and code generation tools that are used broadly. There may be some ideas to learn from that community.<p>JS AST specs: estree [0] and babel's AST [1]<p>Parsers: babel [2], acorn [3], or espree [4]<p>Transformers: babel, recast [5], or jscodeshift [6]<p>Codegen: babel or escodegen [7]<p>[0] <a href="https://github.com/estree/estree" rel="nofollow">https://github.com/estree/estree</a><p>[1] <a href="https://babeljs.io/docs/en/babel-parser#output" rel="nofollow">https://babeljs.io/docs/en/babel-parser#output</a><p>[2] <a href="https://github.com/babel/babel" rel="nofollow">https://github.com/babel/babel</a><p>[3] <a href="https://github.com/acornjs/acorn" rel="nofollow">https://github.com/acornjs/acorn</a><p>[4] <a href="https://github.com/eslint/espree" rel="nofollow">https://github.com/eslint/espree</a><p>[5] <a href="https://github.com/benjamn/recast" rel="nofollow">https://github.com/benjamn/recast</a><p>[6] <a href="https://github.com/facebook/jscodeshift" rel="nofollow">https://github.com/facebook/jscodeshift</a><p>[7] <a href="https://github.com/estools/escodegen" rel="nofollow">https://github.com/estools/escodegen</a>
Haha, the old Java GUI builders in the 90s did something like this. You could either drag around the window (and the code would be updated) or modify the code (within limits) and it would parse it into the GUI builder.<p>Who's old enough to remember the Symantec Visual Cafe IDE?
This is neat! I’m curious if you see this being extended for other languages, or the concept being applied in other projects?<p>As for similar concepts, several projects by builder.io have some overlap. Most notably Mitosis[1], but I’d be shocked if TS-Lite[2] isn’t using similar techniques. Potentially Qwik[3] as well but I’m not sure, I would have bet that’s using Mitosis but it looks like that’s the other way around.<p>1: <a href="https://github.com/BuilderIO/mitosis" rel="nofollow">https://github.com/BuilderIO/mitosis</a><p>2: <a href="https://github.com/BuilderIO/ts-lite/tree/main/packages/core" rel="nofollow">https://github.com/BuilderIO/ts-lite/tree/main/packages/core</a><p>3: <a href="https://github.com/BuilderIO/qwik" rel="nofollow">https://github.com/BuilderIO/qwik</a>
My dream would be bidirectional openAPI. I used it for a project and liked it but after the generators are run, making changes becomes more not less work to keep the spec and code in sync.<p>Something like this would be amazing if it could be integrated as a generator. I'd love to do it if I had the time
What you've done looks pretty smart and definitely worth a deeper look. My main interest is in visual design generating code, especially for animation timing.<p>The concept may be portable - the devil is in the millions of details on which I've seen many promising tools bog down and die.<p>Also, please, don't say _first of its kind_ unless you've done enough research to be confident.
Cool idea! Was wondering if you could elaborate a little on what type of new dev tools would benefit from this. One of the prototypes I worked on was bidirectional code gen for no-code tools but it felt like it might not be the most useful thing.
Have you read/study about Category Theory?<p>I find it incidentally related that you named your project "Monocle" while it does something quite similar to a concept there called lenses, so have a look at it if you have a chance!<p>Btw, really neat idea.
Presumably the results in the reverse direction are sometimes ambiguous, so x1 -> y -> x2.<p>I wonder it there’s a general code improver that tries this transform on every function looking for a shorter x2 than x1.
Looks super cool!<p>For those more experienced in Programming Language Theory, how does code generation slot into PL theory? Is there some kind of common formalism for it?