TE
TechEcho
Home24h TopNewestBestAskShowJobs
GitHubTwitter
Home

TechEcho

A tech news platform built with Next.js, providing global tech news and discussions.

GitHubTwitter

Home

HomeNewestBestAskShowJobs

Resources

HackerNews APIOriginal HackerNewsNext.js

© 2025 TechEcho. All rights reserved.

Data Center Servers Suck, But Nobody Knows How Much

25 pointsby nightbrawlerover 12 years ago

11 comments

jwsover 12 years ago
Just an echo of the NYTimes non-story.<p>Bizarrely, this one self repudiates its own title near the end where they contacted a person that knows what they are doing…<p><i>Over at Mozilla, Datacenter Operations Manager Derek Moore says he probably averages around 6 to 10 percent CPU utilization from his server processors, but he doesn’t see that as a problem because he cares about memory and networking.</i><p><i>…</i><p><i>After we contacted him, Moore took a look at the utilization rates of about 1,000 Mozilla servers. Here’s what he found: the average CPU utilization rate was 6 percent; memory utilization was 80 percent; network I/O utilization was 42 percent.</i><p>CPU use is irrelevant to most internet servers.<p>Over the week, I operate my car engine at about 1.2% capacity. Maybe they should write about that.
评论 #4632773 未加载
评论 #4632361 未加载
评论 #4631925 未加载
评论 #4631615 未加载
评论 #4631602 未加载
评论 #4632373 未加载
ChuckMcMover 12 years ago
Sigh, I wonder why people write stories like this.<p>Data Center servers don't suck, and I'd bet that most folks running them understand what their utilization is. Blekko has over 1500 servers in its Santa Clara facility and we know pretty much exactly how utilized they are, but that is because we designed the system that way.<p>Its funny how things have come full circle. Back in the 70's you might have a big mainframe in the machine room, it was so expensive that the accounting department required you to get maximum use out of the asset, so you had batch jobs that ran 24/7. You charged people by the kilo-core-second to use economics to maximize their value.<p>Then minicomputers, and later large multi-CPU microcomputer servers (think Sun E10000 or the IBM PowerPC series) replaced mainframes they didn't cost as much so the pressure to 'get the return' was a bit lower, you could run them at 50% utilization and they still cost less than you'd expect to pay for equivalent mainframe power.<p>Then the dot.com explosion and suddenly folks were 'co-locating' a server in a data center because it was cheaper to get decent bandwidth there rather than run it the last mile to where your business was. But you didn't need a whole lot of space for a couple of servers, just a few 'U' (1.5" each of vertical space) in a 19" rack. And gee, some folks said why bring your own server, we can take a machine and put a half dozen web sites on it and then you could could pay like 1/6th the cost of 4U of rack space in the Colo for your server. Life was good (as long as you weren't co-resident with a porn or warez site :-)<p>Then, at the turn of the century, the Sandia 'cheap supercomputer' and NASA Beowulf papers came out and everyone wanted to put a bunch of 'white box' servers in racks to create their own 'Linux Cluster' and the era of 'grid' computing was born.<p>The interesting thing about 'grid' computing though was that you could buy 128 generic machines for about $200K which would out perform a $1.2M big box server. The accountants were writing these things off over 3 years so the big box server cost the company $400K/year in depreciation costs, the server farm maybe $70K/year (if you include switches and such like) so it really didn't matter to the accountants if the server farm was less 'utilized' since the dollars were so much lower but the compute needs were met.<p>Now that brings us up to the near-present. these 'server farms' provided compute at hitherto un-heard of low costs, and access to the web became much more ubiquitous. That set up the situation where even if you could offer a service where you got just a few dollars per 1,000 requests to this farm, like a real farm harvesting corn, you made it up in volume. Drive web traffic to this array of machines (which have a fixed cost to operate) and turn electrons into gold. If you can get above about $5 revenue per thousand queries ($5 RPM) you can pretty much profitably run your business from any modern data center.<p>But what if you can't get $5 RPM? Or your traffic is diurnal and you get $5 RPM during the day but $0.13 RPM at night? Then your calculation gets more complex, and of course what if you have 300 servers, 150 of which are serving and 150 of which are 'development' so you really have to cover the cost of all of them from the revenue generated by the 'serving' ones.<p>Once you start getting into the "business" of a web infrastructure it gets a bit more complicated (well there are more things to consider, the math is pretty much basic arithmetic). And 'efficiency' suddenly becomes something you can put a price on.<p>Once you get to that point, you can point at utilization and say 'those 3 hours of high utilization made me $X' and suddenly the accountants are interested again. For companies like Google whose business is information 'crops', they were way ahead of others in computing these numbers, Amazon also because well they sell price this stuff with EC2 and AWS and S3 and they need to know what prices are 'good' and which are 'bad.' It is 'new' to older businesses that have yet to switch over to this model. And that is where a lot of folks are making their money (you pay one price for the 'cloud' which is cheaper than you had been paying, so you don't analyze what it would cost to do your own 'cloud' type deployment). That will go away (probably 5 to 10 years from now) as folks use savings in that infrastructure to be more competitive.
jeremyjhover 12 years ago
Conflating utilization with efficiency is a mistake that is made so often and so reliably that it probably alone explains why reporting on this is sparse.<p>To understand if a system is inefficient, I would have to know if the same computing could be performed at a lower total cost of ownership. The utilization of the actual specific resources is an input, but only to understand the computing demand. The reason it is so common to "throw servers at it" is because the hardware and even the power is very cheap compared to human labor costs; and there are significant labor efficiencies in deploying infrastructure in batches.
gphilover 12 years ago
"After we contacted him, Moore took a look at the utilization rates of about 1,000 Mozilla servers. Here’s what he found: the average CPU utilization rate was 6 percent; memory utilization was 80 percent; network I/O utilization was 42 percent."<p>Sounds like they are stuffing as much data as possible into RAM for performance reasons, which causes the servers to be memory-bound, and as such they have more CPU available than they need relative to the amount of RAM.<p>I am (and I think a lot of other people are) in the same situation.
tluyben2over 12 years ago
This is a political issue as the article already indicates: even though you could just buy very energy efficient, but worse performing ARM based servers with (depending on your needs) networking/memory/IO focus, it is easier to do the 'no-one has been fired for buying IBM' route: if something goes wrong, it's easier to say that you bought the fastest thing money can buy (who cares if it's running at 6% CPU util) than going the 'experimental' route with low energy equipment in servers risking downtime which could be because of your decision.<p>The problem is of course that if you go below the enterprises, it's probably even worse, where high powered servers with cpanel running max 400 accounts per server could be put on a microcontroller costing a few cents on average. I ran servers which had 300-400 accounts on them and where the TOTAL number of sites had a few 100 requests <i>per</i> <i>month</i>. I think this is the case for most of the 100s of 1000s of servers running at hostgator, godaddy etc. The whole problem is 'peaks'. Management will ask you 'nice story, but what about peaks'? So you just buy the fastest thing there is, slap VMWare on and hope for the best. I think power management like in mobile phones would not be that bad for this purpose; most of the time you just switch off all cores but one and power down the clockspeed; when a peak comes, you just power up fully.
评论 #4632427 未加载
JimmaDaRustlaover 12 years ago
I'm sure that this has been said many times, but the "utilization" is not the same as the need to have the availability of the capacity - we could run some servers on a 386 and achieve 90% utilization, but now it takes two minutes to view a web page!<p>Also, CPUs are known to "step down" their clock speed to be power effecient when not under load - this is not included in a percentage of utilization.<p>Also, like someone else mentioned, bigger costs are the cooling techniques. Personally, I can't wait to see stuff like this utilized in the industry: <a href="http://techcrunch.com/2012/06/26/this-fanless-heatsink-is-the-next-generation-in-cpu-cooling/" rel="nofollow">http://techcrunch.com/2012/06/26/this-fanless-heatsink-is-th...</a>
alexchamberlainover 12 years ago
Let us not forget that running servers at 100% CPU/Memory/Network IO (whatever) is bad too. Sudden spikes or a machine crashing will kill your service.
评论 #4632815 未加载
staunchover 12 years ago
CPU performance is simply far beyond what we need in most cases.<p>What's the average CPU utilization of desktops/laptops across the world? I wouldn't be surprised if it was even lower.<p>My laptop:<p><pre><code> Cpu(s): 1.4%us, 0.1%sy, 0.0%ni, 98.4%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st</code></pre>
评论 #4632802 未加载
评论 #4632251 未加载
greenyodaover 12 years ago
Data centers have to be designed to handle peak capacity, not average capacity. For example, if a retail business's web site couldn't handle the peak loads in the month before Christmas without degrading response time, they'd go out of business.
JakeFratelliover 12 years ago
The real issue is the lack of testing on the triple-redundant backup power systems. Utilization is meaningless on a server that isn't turned on.
ollybeeover 12 years ago
Hosting companies are in the business of selling servers. In my experience most customers dedicated servers are hugely underutilised but they only get the higher level of support or service if they take out dedicated tin.