Brightbox vs Amazon EC2: Phoronix Ubuntu 12.04 Benchmarks

You have to careful comparing one specific EC2 instance with other providers. Unlike most compute services, EC2 infrastructure is heterogenous. The c1.xlarge instance compared might deploy to 2 or 3 different chipsets in us-east-1, each with different performance characteristics. Based on the results they posted, this instance appears to have deployed to a 2.13GHz E5506 host. Most likely, Brightbox is using a homogenous infrastructure, and likely something a bit faster, like X55, X56 or E5s. For a more apples to apples comparison, they might have tested against an m2.4xlarge (8 cores) or cc1.4xlarge (8 cores hyperthreaded to 16), which are also 8 cores and deploy to X5550, X5570 or E5-2665. Based on my own benchmarking either of these instances would have performed comparably or better than the Brightbox 8GB instance.<p>The point is, you have a lot of options with EC2 in terms of performance, and comparing a single instance deployment doesn't really do it justice.<p>[Edit] Here are a few links to Phoronix results for EC2 instance types not included in their testing. m2s, cc1s and cc2s are the better CPU performers in EC2 due to faster processors. If you do a CPU performance comparison, these instance types should not be excluded (as I observe they often are):<p>m2.4xlarge - 8 cores - X5550 <a href="http://openbenchmarking.org/result/1208019-SU-CLOUDHARM39" rel="nofollow">http://openbenchmarking.org/result/1208019-SU-CLOUDHARM39</a><p>m2.4xlarge - 8 cores - E5-2665 <a href="http://openbenchmarking.org/result/1208139-SU-CLOUDHARM43" rel="nofollow">http://openbenchmarking.org/result/1208139-SU-CLOUDHARM43</a><p>cc1.4xlarge - 8 cores/16 HT - X5570 <a href="http://openbenchmarking.org/result/1208127-SU-CLOUDHARM93" rel="nofollow">http://openbenchmarking.org/result/1208127-SU-CLOUDHARM93</a><p>c1.xlarge - 8 cores - E5506 <a href="http://openbenchmarking.org/result/1208134-SU-CLOUDHARM15" rel="nofollow">http://openbenchmarking.org/result/1208134-SU-CLOUDHARM15</a><p>These instances were tested using CentOS 6, which will be a bit slower than Ubuntu 12.04.

People keep comparing EC2 and other hosting providers in terms of performance and pricing. However, the biggest value in my mind is related to the agility it provides (you can basically model and control your infrastructure through a programming API) and the whole ecosystem around it (CloudFront, Route53, Beanstalk, S3). Some vendors, including Brightbox, have some of that functionality but it is a very small fraction of what is possible with AWS

From the comments:<p>In your price comparison you forgot to mention EC2 offers reserved instances. Your instance would cost $466 USD a month, and an EC2 reserved c1.xlarge instance would cost $102 a month with a 3 year reserved instance. If you average out the cost to reserve an instance over 3 years, the total would be $188 a month, This means EC2 is actually 60% cheaper than Brightbox.

Posted on brightbox.com.<p>Surely this post won't be biased in the least.

Looking at the pricing reminds me of how expensive cloud computing is if you keep them all running 24/7..<p><pre><code> The small one would cost: server: £0.025 * 24 * 30 = 18.00 ip: £0.0035 * 24 * 30 = 2.52 20 GB /m out: £0.12 * 20 = 2.4 5 GB /m in : £0.08 * 05 = 0.4 = £23.32 </code></pre> Is this estimate correct? For this price you can get four and a half virtual 1-core servers, with 4 ip's, 4 TB of outgoing data and unlimited incoming data.<p>Of course, cloud computing has additional features, so I'm not saying it's never worth it. But it seems like a bad deal for most uses.<p>Am I missing something? or is this just the price you have to pay for the cloud "extras" ?

We found comparing Xen to KVM on the same hardware showed similar results to this too (things doing lots of ram allocations, like redis, do very well with KVM).

If you are going to benchmark Apache, use 2.4 with event MPM which can be as fast as nginx.