Go 1.3+ Compiler Overhaul

While I agree with the end goal here, isn&#x27;t 60k lines of C actually a rather small program to attempt to port in a one shot effort? There will also be an inevitable clean up phase since the Go generated is less than likely to be idiomatic.<p>For me the really fun bit is getting to having the whole runtime in Go, which would be a situation not dissimilar from classic Smalltalk environments - especially writing the GC in the target language.

For comparison, the original Rust compiler was written in OCaml. AIUI, the Rust compiler build now has three stages: a blessed binary snapshot of Rust+LLVM (stage0) compiles and tests a new build (stage1) from source, then (as a bootstrap sanity check) the stage1 compiles and tests a new build (stage2) from the same sources. Needless to say, this is a time-consuming process. :)

This seems like definitely the right decision. If the C# compiler had been written in C# from nearly the beginning I think it would have heavily influenced language and infrastructure development going forward.<p>It&#x27;s much better to be in the position now, when the language is barely used, than to have to support it later.

Besides the immediate appeal of bootstrapped compilers, we&#x27;re also going to have a tool to turn C code into Go code when this is done. I imagine this might breathe some new life into some stagnant projects.

Interesting development, but to be a bit crusty, statements like:<p>&quot;It is easier to write correct Go code than to write correct C code.&quot;<p>and<p>&quot;It is easier to debug incorrect Go code than to debug incorrect C code.&quot;<p>and<p>&quot;Go is much more fun to use than C.&quot;<p>Really are very subjective and don&#x27;t add much beyond setting a POV for reading the article.

I don&#x27;t really get the motivation behind this effort. I&#x27;m not very familiar with the Go compiler, but it seems like make-work.<p>&gt; It is easier to write correct Go code than to write correct C code.<p>But the correct C code is already written.<p>&gt; It is easier to debug incorrect Go code than to debug incorrect C code.<p>Is the C code incorrect? How much debugging is left to be done?<p>&gt; Work on a Go compiler necessarily requires a good understanding of Go. Implementing the compiler in C adds an unnecessary second requirement.<p>The compiler is already implemented in C.<p>&gt; Go makes parallel execution trivial compared to C.<p>Is it not trivial to run multiple C compilers in different processes?

(Update: TLDR --&gt; Modifying the Dick Sites quote from the original article: &quot;I would rather write programs to help me <i>refactor&#x2F;transform</i> programs than <i>refactor&#x2F;transform</i> programs.&quot; So, a programming language highly amenable to that would be very powerful).<p>I just had an insight while reading this. It&#x27;s a powerful insight. (I am surely not the first person to have this insight, but it&#x27;s WAY ahead of being mainstream).<p>A programming language that could be PROGRAMMATICALLY REFACTORED would be a HUGE home run.<p>I know Lispers will jump in here. But I know Lisp (somewhat), and it does not have what I&#x27;m talking about. (Hell, maybe Go does ... if so, I finally get what the fuss is about).<p>The idea is that you could develop a huge codebase in this language, and then you could write code (probably in some other language) that REFACTORS the original codebase. Of course, you CAN write translators, as the original article mentions. But I want a whole new programming language that is designed from the GROUND UP to be amenable to programmatic transformation.<p>NOT just Lisp with its AST&#x27;s and macros. The &quot;transformation language&quot; should be able to understand the following aspects of the original code: its modularity, its test coverage, which parts are functionally pure&#x2F;impure, which parts are parallel&#x2F;not, and the full compiler-level semantics of every piece of text in the code (in other words, what JetBrains knows ... this is a local variable name, this is a function name, etc.)<p>In other words, the original language has to capture more of the programmers intent (probably by inferring most of it). The intent-information is mostly or completely unneeded at runtime, but it is VITAL at automated translation time.<p>Imagine having such a codebase, and being able to pull up a REPL and interactively start changing the modularity of the code by issuing commands. Or telling the transformation system to parallelize some portion of code that wasn&#x27;t parallelized previously. Or saying something like, &quot;Take all the code snippets that instantiate the xyz data structure, and change them to call this function instead.&quot;<p>Don&#x27;t miss my point -- we have features like this here and there. Some IDE&#x27;s more than others, some languages more than others. Nothing new there. But I&#x27;m talking about a <i>new programming language designed from the ground up to be highly amenable to this kind of interactive, automated transformation</i>. In other words, I think this might be the killer feature that allows one programming language to outcompete most others.<p>This might be the one programming language feature that we should focus on now, more than any other.

Go is already extremely fast to compile, I&#x27;m puzzled that they choose to improve this aspect of the compiler, which nobody really complains about, while ignoring elephants in the room such as generics and a proper exception mechanism.<p>It&#x27;s almost as if they choose to tackle the easy problems instead of the hard ones.

&gt; But I want a whole new programming language that is designed from the GROUND UP to be amenable to programmatic transformation.<p>Pretty much any modern statically typed language can easily be refactored.<p>The problem is not the refactoring, it&#x27;s deciding when to refactor and to what (e.g. renaming). This is much more of a human than a computer problem, and as such, it will probably be intractable for a very long time.

I&#x27;m most excited about Phase 4b. The Go compiler currently leaves a lot of performance on the table with its weak optimization capabilities. Having an intermediate representation and a few optimization passes will recover the majority of that performance.

This is really exciting. I can&#x27;t wait to port some legacy C code using it. It should make it a lot easier than doing it by hand.

This explains why there is &quot;goto&quot; keyword in Go spec. Nobody uses it, except state machines and reducers for compilers. They probably thought they&#x27;ll rewrite compiler in Go and didn&#x27;t drop that keyword?

Very interesting. The cat-and-mouse problem mentioned a couple of times - finding a solution that allows development on the old to continue while still working on the new - is very similar to a problem I had in a previous job.<p>We had an Actionscript 2 application that was under constant development and use on the web, and we needed&#x2F;wanted to port it to Actionscript 3 to make use of new language features. I would write a tool that would attempt to translate between the two automatically, and then any more complex transformations could be hand coded and&#x2F;or added to the translation engine.<p>Of course, the project did not succeed during my time at the company, mostly because the scope of the translator was so limited; there were too many concepts in AS2 (the borrowed javascript setInterval, for example) that had no equivalent line-for-line in AS3 (where you had to use the new Timer class to delay&#x2F;loop execution) and therefore couldn&#x27;t be simply substituted. Perhaps if the parser had tried to represent higher order concepts than files and lines of code (e.g. classes, instance data, methods) the project would have got further - but we simply didn&#x27;t have the resource to spend on it, nor the ability to freeze features on the AS2 codebase.<p>As it was, the AS3 version of the app never quite reached feature parity with the AS2 version, which raced ahead too quickly for the new features to be added by hand in AS3 (and too many changes had already been made by hand to make re-running the parser viable.)<p>If Go is indeed simple enough to translate into from (a specific subset of) C, and if Google can afford more than one engineer for a week to write the tool, then this sounds like it&#x27;s off to a better start than my project was.

&gt; There are many corners of C that have no direct translation into Go; macros, unions, and bit fields are probably highest on the list. Fortunately (but not coincidentally), those features are rarely used, if at all, in the code being translated.<p>Sounds like they had this idea in the back of their minds from the beginning, and wrote the C code accordingly.

Alas, that means &quot;Plan 9&quot; bits in go is going to vanish. Luckily some of Plan 9&#x27;s legacies (saner stdio, simpler socket interface, channels) are embodied in go itself and hopefully getting into mainstream.<p>9P was not so lucky (it&#x27;s more of an OS thing than a language thing), but I wish it could take off somewhere.

Noice. Business case aside, that sounds like a fun exercise for those with the chops to tackle it.

The mention of porting the rest of the plan 9 system to Go is the most interesting part of this document. Is this just academic, or does Google actually want a complete new system (kernel and all) implemented in Go?

I don&#x27;t think the tool to automagically convert C code to Go will work. At the very least it will take much longer than expected to get it right. And when it&#x27;s finally ready, he resulting Go code will be very slow, so they won&#x27;t be able to use it without serious manual modification. Meanwhile, the C compiler will continue to add new features, and each one will have to be converted to Go and then refactored for performance. It won&#x27;t work.

Why not start from scratch in go? C and go are different programming languages with different standards etc. Why rely on shoehorning the C version into go?

Fascinating and fun. I really hope perf doesn&#x27;t regress much from this: the zippy edit-compile-run loop is part of why Go&#x27;s fun to use.

&quot;The Go compilers are also significantly larger than anything we’ve converted: over 60,000 lines of C.&quot;<p>Couldn&#x27;t the compiler translation just be crowd sourced by a small group of people? It wouldn&#x27;t be the best result but if people just stuck to the original C design, it would work as a first implementation.

A related article from a few weeks ago: <a href="http://tratt.net/laurie/blog/entries/the_bootstrapped_compiler_and_the_damage_done" rel="nofollow">http:&#x2F;&#x2F;tratt.net&#x2F;laurie&#x2F;blog&#x2F;entries&#x2F;the_bootstrapped_compil...</a>

This are great news. I always defend bootstrapping is the proper way to write compilers for general purpose languages.<p>Looking forward to the day Go&#x27;s runtime is free from C code.

Is translating seriously the best option? Are the translators accurate enough?

Anyone have a non-Google Docs link? It isn&#x27;t loading on my end.

Now if only they&#x27;d do something about that garbage collector.

How would this affect the Plan9 and Dragonfly ports?

So, can Go channels be implemented using channels?

This is the best argument to go D.<p>The D compiler is waaaaay better, written by a compiler expert.