Twitter: From Ruby on Rails to the JVM [video]

I was recently in a quandary over the choice of technology. I started RoR and I really like it. However I was concerned about long term implications of that choice. The thing that I am taking from this talk is that I shouldn't worry about that, right now. If and when I need to scale, I will have enough resources to make a better choice. Resources that I don't have right now.

One thing that wasn't touched was JRuby[1], on their site they state high performance and real threading as advantages.<p>If twitter has (some of) the best ruby developer (mentioned somewhere at the end of the video), why have they neglected JRuby? Why is it no option? For legacy code with native extensions this makes sense. But is jruby slower, more memory hungry on the JVM then scala or clojure? I always though that JRuby was one of the more performant languages on the JVM?<p>Apart from that it was an interesting talk.<p>[1]: <a href="http://www.jruby.org/" rel="nofollow">http://www.jruby.org/</a>

Direct YouTube link: <a href="http://www.youtube.com/watch?v=ohHdZXnsNi8" rel="nofollow">http://www.youtube.com/watch?v=ohHdZXnsNi8</a>

If any of this stuff sounds interesting to you, we're hiring for all sorts of positions: <a href="http://twitter.com/jobs" rel="nofollow">http://twitter.com/jobs</a>

Part of the problem is that any typical Web Application framework is ill suited to building a full real time system like Twitter.<p>I doubt their story would have been much rosier if had they gone with Spring MVC, Hibernate, and Oracle from the start.<p>The minute you start moving away from CRUD based application design and moving into SOA, you're already signing up for a significant rewrite, even if you stick with the same platform.

No one will ever be in the position of handling so much real-time data as Twitter is. The rest of us can just get on with it and stop trying to pre-empt situations that will probably never happen to us.

Many people seem to refer only the scalability (performance, that is) side of the argument but only few who actually pointed out the developer's productivity of choosing Scala and Java for Twitter situation.<p><a href="http://www.infoq.com/articles/twitter-java-use" rel="nofollow">http://www.infoq.com/articles/twitter-java-use</a>

With the release of Java 7 (invokedynamic) the performance of these dynamic languages (like ruby and python) may become much less of a factor (JRuby and Jython). At least that's what the JRuby folks imply: <a href="http://www.engineyard.com/blog/2011/jruby-1-6-released-now-what/" rel="nofollow">http://www.engineyard.com/blog/2011/jruby-1-6-released-now-w...</a><p>Punch Line from Link:<p>"There’s a very real chance that invokedynamic could improve JRuby performance many times, putting us on par with our statically-typed brothers like Java and Scala. And that means you can write Ruby code without fear. Awesome."

I noticed that every time Twitter acquired a company, they also accrued a new language:<p>* Summize brought Scala<p>* BackType brought Clojure<p>Has anyone noticed that pattern elsewhere?

I think most people misunderstand the point made by Raffi...yes, JVM has awesome performance, but never try to solve performance issue up front by sacrificing the agility offered by RoR. Not everyone is building Twitter, you don't even know whether you will hit the point where the VM is blocking your way. If you blame all the performance issue to the VM level, you are simply doing it wrong...In 90% of the cases, MRI is fast enough and meet your requirement (GitHub, Groupon, Living Social and many others are using RoR BTW). Twitter is very pragmatic at this respect, they have tried all the means to scale the app on RoR before they move to the JVM. Never ever try to solve a requirement that doesn't even exist in your own app...

I'm still not getting the futz they're making over their "scale".<p>So your inbound load is 7000 tweets/sec or roughly 250 MBit/s (assuming 4k per tweet). Then you fan that out to (assuming) 20 append-only mailboxes on average.<p>Perhaps my assumptions are far off, but I'm only arriving at a couple GBit/s here and a low two digit number of terabytes/storage per year.<p>This sounds like "a couple racks" to me, not like "a couple datacenters".

very interesting talk. I was surprised that there was no mention of using a lower level language i.e. c or c++ in order to maximize cpu/ram utilization. while JVM is a clear winner over ROR it does add some overhead. I guess it is a sweet spot between performance and code manageability.

Any thoughts solving scaling issues with node.js..

Anyone pondering their technology stack should watch this video. It doesn't matter what you initially employ as a technology/framework/server to get your app up and running, but if you need to scale the JVM is were it's at. I say that as a Rubyist.

is there any good site that would explain when to use what technology - like<p>when you need many concurrent users then dont use ROR but use JVM or C++ or Scala instead,<p>if you need to build a fast prototype build on ROR or PHP etc.

An ignoramous question: Aren't there faster languages than Java?

I'm pretty sure the project I'm working on will be the next Twitter, and this video's making me second guess staying with RoR.