The sorry state of server utilization and the post-hypervisor era (2013)

The article is looking at it all wrong. To solve a problem, start by looking at those that already solved it. Then, see if you can apply that. Mainframes have long had ridiculously high utilization and throughput. Secret is their I&#x2F;O architecture: computing happens on compute nodes and I&#x2F;O is managed by I&#x2F;O processors, both of which are well-integrated. If Intel etc copy this, they&#x27;ll get much higher utilization and throughput. Smart embedded engineers do the same thing albeit with microcontrollers.<p><a href="https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;I&#x2F;O_channel" rel="nofollow">https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;I&#x2F;O_channel</a>

It&#x27;s 2015, not 1985. Most people are not paying IBM for every CPU cycle (used or not) on a mainframe. Should IT staff try to look &quot;good&quot; on a &quot;CPU utilization&quot; report that belongs in a history book by buying lower-end hardware, or should they spend a tiny amount of extra money to ensure that customers get good performance during peak periods?

These aren&#x27;t really startling findings. Most apps in the enterprise require separate instances for development, staging, production, and a hot standby for continuation of business. And you need each of those environments for multiple tiers (db, app server, etc). And you need the entire stack replicated to each local datacenter because of latency (so the idea of having the APAC users use the database at night and the NAM users use it during the day just doesn&#x27;t work in practice). So a typical business app can easily require &gt;10 server instances, most of which will sit idle most of the time.

&quot;A post-hypervisor world &quot;<p>lol. I&#x27;ve been predicting a backlash against this virtualization hype since 2005, and this is the first time I&#x27;ve heard anyone else mention anything like it.<p>Of course, if you had told me in 2005 that we would be switching from hypervisors back to containers, I would have broke down crying.<p>Is our industry run by masochists? or just the inexperienced, who don&#x27;t know any better?

If the issue is running out of memory before running out of CPU times, then containers wont help much, apart from to the extent that memory is overallocated with static amounts to vms. The solution is either larger memory systems, which are now much more widely available since this article was written, or using less memory for applications.

Looking at this as an environmental problem makes some sense. I used to rent cheap hosted servers and moved to virtualized systems like AWS, Azure, and AppEngine partially because of environmental impact and partially out of convenience.<p>We need staging servers and redundant backups so getting really high utilization is not possible but I hope to see a lot of improvement.<p>The big companies seem to be doing things better. At Google, I had a bit of angst running 10k processor jobs but they do use solar, set up data centers near hydro electric sources, etc. Same as Amazon, Microsoft, etc.

Well, over provisioning is good for perceived performance. Let corporate it increase efficiency in the same ham handed way and you might find low latency needs a new advocate.

The idea that Google was industry leading on non-batch loads in 2013 seems wrong to me. They were not selling those services then, so they did not have a positive profit motive to optimize that usage (only a motivation to cut costs, which I&#x27;m told is not nearly as effective). Amazon has had that motivation (and necessity with their non-existent margins in every other part of their business) for long enough to actually accomplish something.

But is average utilisation the right metric? The work day is only like 8-10 hours, I would expect many corporate infrastructures to be only active during that period. Plus you don&#x27;t size your infrastructure to a typical workload, you size it to be able to accommodate higher than usual peak workload otherwise you will be down at the busiest period.

Which of these hypothetical situations is more realistic?<p>CEO: &quot;I see that we had 99.994% uptime for the last six months, and we came in very close to the forecasted budget. Well done, engineers!&quot;<p>CEO: &quot;I see that we had 99.9% efficient usage for the last six months, and we reduced our budget. Well done, engineers!&quot;<p>Neither scenario is realistic, of course. Uptime is nice and efficiency is nice and budgets are nice, but what the CEO is actually interested in is:<p>VP Customer Support: &quot;Our satisfaction rate is up, call quality metrics are great. I looked over the call stats and it looks like we&#x27;re no longer getting complaints about performance or unreachability.&quot;