It's not really "go" that makes the difference. it's how the runtimes and frameworks are used and/or made.<p>frameworks on top of frameworks, all being over engineered, with poor understanding of what the system actually does, result in super slow apps on top, that you generally go to aws to scale.<p>It's not the first time that I see people reducing a dozen servers that were "always maxed out" by a couple of servers "that can barely feel the load".<p>Yeah throwing hardware at the issues is fine'n all but we're been way over the limit too many times in too many directions.<p>Go provides a clear start/APIs/framework, and the language enforces good habits. Also, it doesnt have things like global interpreter locks.<p>The lesson? Stop using cool techs because there's blogs about them. UNDERSTAND the tech before using it.
Go's definitely picking up momentum. I know that Mozilla is shifting much of its services infrastructure to Go.<p>We're porting over our Arabic sentiment engine, currently in Python/Cython to Go. If you're dealing with simple data structures, going from Python to Go is almost a line-for-line port, but the performance benefits are, of course, massive. Our benchmarks show a 40x speed improvement so far.<p>Lastly, for anyone thinking about taking the plunge, use Go tip (from their source code repo) - don't use their "stable" releases. They fix bugs so fast you'll always want to be current.
I went briefly to through the comments, it seems that no-one is bothered with lack of details in this post, things are not logical at all, almost like it was written by PR agency in charge of promoting Go.<p>Except it is established business so I would assume things are true, but real motives are hidden. Go is obviously good language for some specific things, but it's not like ruby is pure trash, how come you did 'everything' in ruby, and then didn't know about eventmachine, but just had to rewrite things in Go.<p>And rewrite happened overnight, right?<p>A lot is missing from this story. I will definitely look more into Go, mostly because someone compared it in comments with TurboPascal, and I have fond memories of Borland tools, Pascal especially.<p>Go and Rails are so different that there is almost no point in comparing them.
I was originally very excited about Go when I first learned about it. But then I got tired and frustrated quickly after having to listen to the other Gophers telling me that I don't need this or that feature because there is a better way to do it in Go.<p>Like.<p>I don't need exceptions because Go function can return multiple values.<p>I don't need a mocking framework like Mockito because Go has interfaces.<p>I don't need an interactive debugger because I can debug with command line using gdb.<p>I don't need named arguments because I can instantiate a struct and call my function with it. (Have you seen Ruby or Javascript code? Almost every function takes 'opts' as a single argument. Go is probably going down this path too.)<p>Then I learned about Scala. I'm not saying that Scala is better than Go. However, it has everything that I need. :)
For large projects, it'd probably make more sense to rewrite the critical code sections in C. rubyinline makes this really easy, and ruby C extensions aren't that bad either.<p>One time on a contracting gig I had some code that was interacting with cairo drawing and imagemagick. They didn't have a compatible raw image format at the time (one was RGBA and the other was ABGR, IIRC.) It's trivial to convert between the two in ruby, but it was taking 60 seconds per image. Once I switched that one loop to C, it took 0.02 seconds.<p>I've got nothing against Go, just suggesting an alternative that might make more sense in some situations.
I've seen a lot of dumb Rails server configs and even dumber usages of Rails without any tuning at all.<p>With a couple weeks or a month of work I could shrink the hosting fees or resources consumed by a factor of 5 out of most Rails apps that are sitting up around 30 servers... and probably a factor of 10. Leaving aside whatever rookie or even intermediate mistakes were made in their Ruby code or their database, this post indicates a lack of understanding of what happened when their server fell over. Proper tuning of a deployment should not trigger a 100% failure mode like this.<p>These folks were itching to get off of Ruby for whatever reason... after all their roots were in Java. If your goal is to do a rewrite and learn a new language and gain some notoriety why waste time learning what you did wrong with Ruby or your server config?
I'm getting the same experience here by entirely rewriting an old PHP web application [1] in Haskell using the Yesod web framework.<p>I've been using JMeter to benchmark both versions of the application. On a 10€/month dedicated server [2], the Haskell one was able to generate 220 dynamic pages per second [3] whereas the PHP one tops at 35 pages per second on a equivalent page.<p>Moreover, concurrence capabilities of Haskell are also pretty sweet : while I was benchmarking the web app using 2,000 concurrent connections, the application server was only using around 90MiB of RAM. I was not able to increase the number of concurrent connections as the client application I was using started to kill my quad core desktop, I suspect Haskell to be able to manage A LOT MORE concurrent connections as I didn't see any decrease in the throughput of the application as I was increasing the number of concurrent connections.<p>[1] <a href="http://files.getwebb.org" rel="nofollow">http://files.getwebb.org</a><p>[2] <a href="http://www.kimsufi.com/fr/" rel="nofollow">http://www.kimsufi.com/fr/</a><p>[3] Static content has been ignored as modern servers like NGinx seem to be able to carry the static content (CSS, images, ...) at more than 5,000 req/sec on the same machine.
Interesting to hear of Go being used in production. It'd be great to hear some more details on your setup when deploying the go processes - how are you managing failover, what's your load balancer, and how are you handling swapping out processes etc? Are you compiling on the server or local machines before deployment? Most other languages have lots of solutions on the deployment side now but Go is so new there isn't much info out there.<p>Also, as you are running an API presumably your app in Rails was pretty simple, did you have the impression things would be more complex in Go if you were writing an app with an extensive front end and UI and using sql? I'd miss all the view helpers etc available at present I think. Going from 30 servers to 2 certainly sounds like a huge improvement, so it was definitely worth it for you, are you thinking of writing any front-end apps in Go?<p>I've been playing around with Go recently and it is a fantastic language for someone coming from dynamic languages like Ruby. I particularly liked interfaces as a way to define a contract for implementations to follow, and the simple package system which encourages you to make your code modular.
Ok, this bit has left me confused.<p>They were Java devs that liked ruby, they wrote applications in Ruby on Rails and the ruby apps were hitting limits so they immediately started looking at other languages.<p>But they don't mention the most obviously (to my mind) simple option.<p>JRuby<p>It is ruby (They like ruby). Most ruby apps can be run on JRuby with very very little changes (No need for a big rewrite) and it runs on the JVM with which they are familiar and it is very fast (and true multi-threading).<p>Maybe they did look into it but I would have thought that would be #1 on their "We tried this but discounted it" list.
> "We also weren't sure if we would be able hire top talent if we chose Go, but we soon found out that we could get top talent because we chose Go."<p>I feel[1] that a smart/talented C/C++/anything developer can go from someone who has never seen or heard of golang to a proficient and productive Go developer in a matter of a few weeks, maybe even _days_, if not less.<p>That's how long it takes to go through the following materials (and fav some for later reference) and play with the language a bit.<p><a href="http://golang.org/ref/spec" rel="nofollow">http://golang.org/ref/spec</a><p><a href="http://www.youtube.com/watch?v=ytEkHepK08c" rel="nofollow">http://www.youtube.com/watch?v=ytEkHepK08c</a><p><a href="http://commandcenter.blogspot.com.au/2012/06/less-is-exponentially-more.html" rel="nofollow">http://commandcenter.blogspot.com.au/2012/06/less-is-exponen...</a><p><a href="http://tour.golang.org/" rel="nofollow">http://tour.golang.org/</a><p><a href="http://golang.org/doc/install" rel="nofollow">http://golang.org/doc/install</a><p><a href="http://www.youtube.com/watch?v=XCsL89YtqCs" rel="nofollow">http://www.youtube.com/watch?v=XCsL89YtqCs</a><p><a href="http://golang.org/doc/code.html" rel="nofollow">http://golang.org/doc/code.html</a><p><a href="http://golang.org/doc/effective_go.html" rel="nofollow">http://golang.org/doc/effective_go.html</a><p><a href="http://golang.org/pkg/" rel="nofollow">http://golang.org/pkg/</a> - use as reference<p><a href="http://www.youtube.com/watch?v=f6kdp27TYZs" rel="nofollow">http://www.youtube.com/watch?v=f6kdp27TYZs</a><p><a href="http://talks.golang.org/2012/splash.article" rel="nofollow">http://talks.golang.org/2012/splash.article</a><p><a href="https://gobyexample.com/" rel="nofollow">https://gobyexample.com/</a><p><a href="http://golang-examples.tumblr.com/" rel="nofollow">http://golang-examples.tumblr.com/</a><p>And a some more similar things that you can mostly get to from golang.org site. The beauty of how concise the language and even its website are, is that you can literally just go through everything there one thing after another.<p>[1] This is my personal opinion based on playing with go the last few weeks/months. I'd love to verify this theory. It's not yet the primary language in which I do things in (I use C++11 atm), but for all my side tasks[2] it proved to be indispensable. And I found it very easy to pick up. I can't wait until I start doing all my work in Go, that will be a true test of its productivity efficiency.<p>[2] <a href="https://gist.github.com/shurcooL" rel="nofollow">https://gist.github.com/shurcooL</a>
Excellent success story with Go.<p>Love this quote... "We also weren't sure if we would be able hire top talent if we chose Go, but we soon found out that we could get top talent because we chose Go."
“the entire process started up with only a few hundred KB's of memory (on startup)”<p>There is no way this could be true: last time I checked, a bare minimal Go HTTP server requires at least 2.8MB of memory on 64-bit machine. Are you using the default net/http library?
This story sounds mighty familiar. I realize the topic is about one language over another, and most of the comments here seem to be focusing on that, but there's another part which I find fascinating: the cascading failure mode when they'd lose a web head.<p>"This would in turn cause the load balancer to think it failed and take it out of the pool, thereby applying the load that the unresponsive server would have been handling to the remaining servers. And since the remaining servers are now handling the load of the lost server plus the spike, inevitably a second server would go down, the load balancer would take it out of the pool and so on."<p>It was 8 years ago, and the customer was running PHP, not Ruby, but otherwise it's the same basic story that lead to the creation of "Surge Protector", or its actual name, "Suicide Pact". <a href="http://rachelbythebay.com/w/2011/06/28/sp/" rel="nofollow">http://rachelbythebay.com/w/2011/06/28/sp/</a>
Why did go "win" over erlang, scala, and clojure? Because it's becoming trendy and "fun?"<p>And, why are "fun" and "joy" terms ruby bloggers use for languages? Is this code for "easy" and "familiar?" As in, "this language is easy to learn.. It does not require us to learn difficult but mind-expanding concepts to become proficient."<p>Replacing ruby/rails with something (just about anything!) results in far fewer servers. Isn't this obvious to all? Is this really news to the average HN reader?<p>If that's an I/O bound API server, bet those two servers could be brought to one in a language/runtime that's more productive but a little less "fun" to learn.
I rewrote my DNS checking tool (<a href="http://www.dnsinspect.com/" rel="nofollow">http://www.dnsinspect.com/</a>) in Go and I saw huge differences in resource usage (previously it was implemented with Ruby on Rails + EventMachine), my memory usage went down from 128MB per background worker to a few KB.<p>Now I'm able to run hundreds to thousands of concurrent reports using a small VPS, the Go application is using 36MB of RAM (24MB front end + 12MB for background workers). Go language is well suited for my particular case (many concurrent IO operations).<p>This was my first Go project, in a week I was comfortable with it, I had many alternatives but I really liked the simplicity of Go, fast compilation and easy of deployment. Because I've missed some pieces from the Ruby's world I've combined Jekyll (Compass, HAML SASS, RedCarpet, etc) with Go. :)
I'm missing the connection between poor API performance under Rails and the decision to do a ground-up rewrite. Was the initial performance problem due to an API problem or a platform problem? Was there any profiling of the poorly performing API server? I'd love to hear the story behind that analysis and how it impacted the tale told in TFA.<p>I ask all this since I've lost count of how many times I've participated in or witnessed an averted rewrite via a good dose of profiling and a few key bugfixes. I'll acknowledge that's not as fun as a clean-slate project, but <i>vast</i> amounts of engineering time were saved.
I agree with the 'Go is fun' (and easy to pick up) sentiment. I spent a few weeks recently writing an NTLM library for Go for a consulting client (I hope they'll let us open source it at some point). All the parsing and bit manipulation was <i>extremely</i> straightforward due to the excellent standard libraries and I was able to quickly move from not knowing Go, to a fully implemented NTLMv1/v2 library!
Another day, another Go PR post on HN.<p>What is new about porting code running in one of the slowest interpreters around to a compiled implementation of another language?<p>I was already porting Perl/TCL code to C++ back in 2002 with similar performance results.<p>Don't kids nowadays learn anything about performance in their CS degrees?
It's an interesting tale, and I honestly have no bias toward Ruby or Go, but the cause/effect relationship is poorly illustrated (we switched languages). We all know runtime / performance / scalability is more complex than that. I'd like to hear where the bottleneck(s) were and how Go solved them.
Last month I picked NodeJS and build couple of sites with it. This month, I want to pick up something new. I was hesitating between Python and Go. Can you answer couple of questions about Go from someone coming from NodeJS:<p>1- What is the state of the external Go libraries, especially DB (MySql), caching libraries (memcached), protocol libraries (Oauth). Are they stable 100%<p>2- How easy is logging and tracing in go?<p>Thanks.
Has anyone had an experience of running websites from Go; or more specifically, how you handle none-HTML content?<p>I've been considering porting my CMS from mod_perl to Go, but I'm not sure how you work with the other files (CSS et al). I did read somewhere that you run Go from Apache but it's not recommended.
It's funny that they argue that they dumped the JVM derivatives like Scala, but wouldn't exactly tell why, nor support their argument with any sort of data/logs.<p>I am genuinely curious to know why they chose Go over Scala. If it was the syntax, etc. I can partially agree because it's one of scala's weak points, but then they pitch the main reason citing performance, so I'm genuinely curious to know.
"We decided to rewrite the API" and now it is over 300x faster. Go had nothing to do with this, it was obviously some boneheaded original design. Probably something like fork+exec on a Ruby VM on each request to enter a chroot.<p>I know HN has a hard-on for Go but come on.
I've read this domino-effect on server clusters on several HN postmortems and I've seen various flavors of this on our own web servers. I'd like to think there's a simple configuration in most servers that prevent 100% cpu utilization from taking place and preventing the server from telling its cluster that it's still alive. Anyone have any experience with this?
Can you describe your workload a bit as well as any benchmarking you did to determine (and perhaps optimize) hotspots in the ruby code before you started the port?<p>Personally, I've been interested in moving to Go on a python-based project of mine. Thus far, I've avoided it because 1) the extra work required to self-implement a few third party libs I rely on and 2) I've been able to eek out sufficient performance using c extensions and cython.
Science and engineering from Bell Labs behind Go is what make this possible. It is not a "different language", it is carefully selected design decisions and ideas behind it.<p>It relies on a principle of being good-enough, not to stuff everything in, as a "feature sellers" and "buzzword shouters" used to do.<p>There are also lot of work of great minds behind Lisps or Erlang, and same principles in a foundation.
"So then the decision came down to which language to use."<p>I find this tendency to constantly jackknife between talking about languages and talking about frameworks a little dizzying.<p>Shouldn't the decision have been what web stack to use? I'm sure you could get half way there with a bespoke Ruby web stack, maybe built on top of JRuby or with a sprinkling of C extensions. You could get the same poor scalability with some badly designed framework on top of Go [1].<p>There's a lack of meat in the post (hence why people are calling it out as PR). How do these features of Go make it easier to write applications or frameworks that cater to this particular scalability need? Could extra effort be put in up front to get something similar out of Ruby or any of the other languages, or will they always be sub-par? What is it about Go that makes it light up the rest of the stack in a way other languages don't?<p>Or have you just traded one trendy technology for another because it promises to be the magic bullet for your current itch?<p>[1] This isn't saying Rails is badly designed.
Hey that's my photo of the GOpher! :)<p>[1] <a href="http://www.flickr.com/photos/jianshen/8080852738/in/photostream" rel="nofollow">http://www.flickr.com/photos/jianshen/8080852738/in/photostr...</a>
OK, I am sold on Go as my next language for web applications. Unfortunately, I can't throw away the existing PHP/MySQL code. Any suggestions on how to integrate Go and PHP into a single web server? Should I be using FastCGI for Apache to call the Go programs?
Out of curiousity, I would be interested to know which version/patches of ruby you were using. With ruby I'm used to running out of memory on servers long before running out of CPU. Of course, most of our workloads are IO bound, so that may be the diffent.
I know I'm late to the party, and no one will read this, but ...<p><pre><code> "... Java derivatives like Scala and Clojure ..."
</code></pre>
Someone help me out here: In what way is Clojure a Javav derivative?
How about BDD, test-driven development and quality insurance? Does Go provide an ecosystem that supports agile refactorings that are common in lean startups?<p>You can say anything bad about the performance of Ruby and Rails etc. but rspec, cucumber, capybara, vcr, factorygirl are really important features to start from zero and reach a viable product.
I wonder how far they could have got going more of a bare-bones Sinatra app using JRuby. It's still not nearly as fast as Go obviously, but take away Rails and some of the more dynamic usages of Ruby on a faster VM and I'd venture to guess you might have been able to go from 30 servers to 10-15?<p>That's still not 2 mind you...
There's simply no technology that makes you switch from 30 machines to 1 (you say you keep 2 just for reliance).<p>Obviously there must be other architectural changes in play here (I'm guessing RoRs thread-per-request vs some kind of event loop on Go?). Please be more specific.
There are a lot of things that can be said about the "average" programmer in such-and-such languages, what they do and don't do, what habits they have, etc. But I'm not interested in working with merely average people.
If you look for a Golang programmer today, it's more likely you'll find a <i>true programmer</i> instead of a <i>me too</i> programmer. So if you're a true programmer, learn Golang. ;)
Now I know that the dynamic languages such as ruby and python are not known for performance, but the benchmarks I have seen with Go look like it isnt exactly the fastest language available either. After reading this article I am more apt to think that something is/was VERY wrong with their Ruby code than to think that Go is that much better. Maybe my lack of experience with either GO or Ruby is showing here, but Ruby cant be <i>that</i> slow can it????
p.s. come disrupt our business: with nearly no hardware, no costs, and no maintence burden you too can pass OMG huge amounts of message traffic.<p>this is one bold as fuck post.
<a href="http://www.dartlang.org/support/faq.html" rel="nofollow">http://www.dartlang.org/support/faq.html</a><p>>> Q. How does Dart relate to Go?#
Dart and Go are both language projects started at Google, but they are independent and have different goals. As a result, they make different choices, and the languages have very different natures, even while we all try to learn from each others' work.<p>Hmm, don't think I want to bet the farm on that.
<i>We decided to rewrite the API. This was an easy decision, clearly our Ruby on Rails API wasn't going to scale well and coming from many years of Java development and having written a bunch of things that handled tons of load with way less resources than what this Ruby on Rails setup could handle, I knew we could do a lot better</i>
Hmm, wouldn't it be better to rewrite the ruby services to Go, the actual workers and not the entire API?
I don't think the API HTTP requests where your main issues here...
Would be great to hear more specifics of the rewrite. What did the API do, what pieces were built in RR, what got replaced in Go, etc.<p>At least as much as you're willing to share. :-)
Man, what is it about Go that just brings out the pricks of HN?<p>There are over three throwaway accounts created exclusively for this thread to shit-post about Go, to steal an old term I have never, ever used on HN before.