Comparing Go and Java, Part 2 – Performance

Please, if you're writing req/resp benchmarks, please include 98/99/99.9% latencies. These are essentially the only numbers that actually matter.<p>At the mean, at concurrency 50, we have things like 37.5ms vs. 12ms. To a user, that is "instant" vs. "instant". Sure, if we stack a bunch up on a page, maybe we'll start to care, but..<p>Much more directly, in real-world scaling scenarios, it matters far more if the 98% number is 2s or 248ms than the mean is 37ms. A 2s latency means 2 out of every 100 requests (and likely &#62; 2% of page views), the user experience will suck. And, all it takes is 6 requests to ensure that 25% of users experience this (your homepage alone probably requires 6 requests).<p>I'm not just raising this to be pedantic--I've seen plenty of systems that have been tuned to do very well on things like req/s or even mean latency but do very poorly for 1 in 100 or 1 in 1000 (vs. being a very fair scheduler at the cost of overall throughput). We reward these systems by measuring and praising the wrong thing. (e.g. mongodb vs. riak)<p>edit: (btw, not intended as a specific defense of either go or java wrt the article, just a general statement about benchmarking these systems)

The article mentioned DropWizard (covered more in part 1). DropWizard is a very nice (the best?) way to write RESTful web services in Java. Coda Hale, that author, has glued together some of the best Java libraries with very sensible configurations, almost to the point of saying "look stupid, this is how you're supposed to do it (in Java)". The application configuration patterns (ugh... I said "patterns"), clear separation of resources, built-in metrics, and health checks is worth studying (at least for journeymen like me).<p>Check it out: <a href="http://dropwizard.codahale.com/" rel="nofollow">http://dropwizard.codahale.com/</a>

No surprises here. Google did a benchmark on go, java, scala, and c++ that's worth reading. <a href="http://www.readwriteweb.com/hack/2011/06/cpp-go-java-scala-performance-benchmark.php" rel="nofollow">http://www.readwriteweb.com/hack/2011/06/cpp-go-java-scala-p...</a><p>Here's my personal experience. Simple go code is actually comparable to c, even for non-io bound tasks. I was very surprised when OpenSSL's AES implementation and go's AES implementations performed similarly in my microbenchmarks. The jvm actually performs very well (go figure that over a decade of optmization in running enterprise workloads would result in a fast runtime). I've inspected the generated assembly in a go code and compared it with that from the equivalent c code and there's no doubt go isn't the most efficient language ever created.<p>Use go if you want something that's (pretty darn) fast, productive, and has a standard library written by some of the most respected names in the field. Don't use go if you need the most mature and performant runtime and libraries. I have no doubt that they will get there eventually.

I'd make a guess in absence of code.<p>Since none of the implementations were IO bound (unsaturated link), I'd bet on memory management as the deciding factor (it's hard to assume that an authentication service will be CPU bound).<p>Then it's not difficult to see why Java won. When it comes to garbage collectors, JVM has the best (production) implementation of GC bar none.<p>Go runtime has a lot of catching up to do.

Does anyone else find 4 lines of error checking (log and panic) boiler plate code for every line of functional code a bit tedious?

That seems reasonable to me - Java's had decades of optimisation, but Go's a fairly new language and runtime.<p>I've just started playing around with Go, I'm quite liking it. It's a different approach to things, which is always fun.

Is this right? Java really has better latency than Go right now? And with 2 processors, Go's latency is 25ms for 50 concurrent requests. Anybody with better experience care to comment on this?

It'd be fun to throw Node.js and Erlang into the mix.

Here's an insightful comment from the blog page, about cpu and memory usage:<p>I downloaded patricks fork and got 4300r/s with maxprocs 1 with the go version, out the box.<p>Recompiling and running with gomaxprocs=100, i got 4400r/s.<p>mvn deploy and running auth-1.0.jar on JDK 1.7 on my box similarly peaked out at 3100r/s.<p>It's worth noting though, that I was using patricks httperf bench.sh modification, and it appears that httperf was cpu bound in both cases, with the kernel and postres taking about half a core between them.<p>Using wrk, by contrast, spun main (the go program) up to 2.5 cores, and 6000r/s. Java under wrk lit all cores for a time, then hit `Exception in thread "async-log-appender-0" java.lang.OutOfMemoryError: Java heap space` `Caused by: ! org.postgresql.util.PSQLException: FATAL: sorry, too many clients already`<p>A bit more investigation and tuning later, using 10 clients `wrk -c 10 -r 100000 -t 4 -H 'Authorization: basic apikey_value' <a href="http://localhost:8080/authenticate`" rel="nofollow">http://localhost:8080/authenticate`</a> Java was spinning out 10kr/s. Go was limited to 6kr/s. It's worth noting some more details however: Go only ever spun up two postgres forks, whereas java spun up 10. There's scope for optimization there. Go used 8mb of ram, whereas java was sitting on 130mb after a few runs. The Java version, cranking out at 10kr/s was maxing the kernel out on one core, so that's probably approaching the practical limit for single machine tests.<p>I suspect putting a load balancer in front of a couple of instances of the go program would allow you to totally smash the java performance, given that it lit 2 cores at 6kr/s, and java lit 8 at 10kr/s. The memory usage tradeoff is significant - JVM sitting at 130mb and Go sitting at 8mb. Clearly everyone needs to draw their own conclusions on this, for their own purposes. The JVM solution is carrying a stats server, a ton of other tooling and so on. The Go system has some - pprof was included, but it's limited by comparison. Arguably gdb and so on can actually be of real use in the Go case, but that's also an exercise for the reader.<p>Interesting any which way you look at it. There are a lot of other interesting side effects (that need working out) in both programs as evidenced by this simple testing. Postgres also needs some tuning if you really want to slam this with anything remotely resembling a real world scale test.<p>My tests were done on OSX 10.8 12B19 on a 2.3GHz i7 wiht 8GB of DDR3 at 1333MHz, an intel 510 SSD, totally untuned postgres. The machine is a Macbook from whatever year that is.<p>Here's wrk, for anyone looking for it: <a href="https://github.com/wg/wrk" rel="nofollow">https://github.com/wg/wrk</a>