Rules of optimization

&gt; <i>But if performance work has been neglected for most of the product development cycle, chances are that when you finally fire up the profiler it’ll be a uniformly slow mess with deep systemic issues.</i><p>Totally true, and I have observed it IRL on large, old projects whose architecture was hostile to performance. And these products were doing computationally intense stuff.<p>&gt; <i>Performance is everyone’s responsibility and it needs to be part of the process along the way.</i><p>Yes.<p>Everyone repeats &quot;only optimize after profiling&quot;. It&#x27;s true: if your program is running too slowly, you should always profile. But that rule only applies when your code is already slow. It doesn&#x27;t actually say anything about the rest of the development process.<p>Developers should have a solid grasp of computer hardware, compilers, interpreters, operating systems, and architecture of high performance software. Obliviousness to these is the source of performance-hostile program designs. If you know these things, you will write better performing code without consciously thinking about it.<p>If this knowledge is weighed more heavily in the software dev community, people will put in the effort to learn it. It&#x27;s not that complicated. If the average dev replaced half of their effort learning languages, libraries, and design patterns with effort learning these fundamentals, the field would be in a much better place.

1999 called. It wants its yellow text on a blue background back. (On the other hand, today&#x27;s hipster medium grey text on a light grey background isn&#x27;t an improvement.)<p>Knuth made his comment about optimization back when computing was about inner loops. Speed up the inner loop, and you&#x27;re done. That&#x27;s no longer the case. Today, operational performance problems seem to come from doing too much stuff, some of which may be unnecessary. Especially for web work. So today, the first big question is often &quot;do we really need to do that?&quot; Followed by, &quot;if we really need to do that, do we need to make the user wait for it?&quot; Or &quot;do we need to do that stuff every time, or just some of the time?&quot;<p>These are big-scale questions, not staring at FOR statements.

A lot of optimizing advice from old misses some of the realities of old. Things generally ran slow. Performance problems were much more in your face and very common. There was a wealth of &quot;tricks&quot; to do with performance, and people made code difficult and brittle in the name of optimization tricks. So there was a bit of push back and a lot more focus put on getting things correct &#x2F; simple &#x2F; well designed. But, because things were slow, and you often didn&#x27;t need to profile, because often things were in your face slow. So you still needed to get it fast, and while you may not have jumped to tricks too quick, you were still quite well aware of designing for performance. You had to consider data structures carefully.<p>These days, things are often just not so in your face and a lot of &quot;don&#x27;t worry about it&quot; advice. Which is often not bad advice in terms of getting things working. But eventually you do find you have to worry about it. Sometimes those worries happen far too late in bigger projects. So I think these Rules are pretty good. I&#x27;d also add in benchmarking very simple aspects of the toolset you are using to get an expectation of how fast things should be able to go. Often I have found (especially in the web world) someones expectation of how fast they think something can go is way too low because they have already performanced bloated their project and think its approximately normal.

This matters. I write emulators. (join me: <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=17442484" rel="nofollow">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=17442484</a>)<p>If you want to run something like Windows 10 in a full-system emulator that is something like valgrind, performance matters. For each instruction emulated, you might need to run a few thousand instructions, and you can&#x27;t get magical hardware that runs at 4 THz. Right from the start, you&#x27;re deeply in the hole for performance.<p>Consider the problem of emulating a current-generation game console or smartphone. You can&#x27;t give an inch on performance. You need to fight for performance every step of the way.<p>Just recently, I did a rewrite of an ARM emulator. It was all object-oriented nonsense. I haven&#x27;t even gotten to profiling yet and the new code is 20x faster. Used well, C99 with typical compiler extensions can be mighty good.<p>It&#x27;s important to have a feel for how the processor works, such as what might make it mispredict or otherwise stall. It&#x27;s important to have a feel for what the compiler can do, though you need not know how it works inside. You can treat the compiler as a black box, but it must be a black box that you are familiar with the behavior of. When I write code, I commonly disassemble it to see what the compiler is doing. I get a good feel for what the compiler is capable of, even without knowing much about compiler internals.

&gt; Not every piece of software needs a lot of performance work. Most of my tweet should be interpreted in the context of a team within an organization, and my perspective comes from a rendering engineer working in game development.<p>Tempers his universals a bit.<p>In general, when working on web apps, which is mostly what I do, you don&#x27;t gotta be quite that ambitious I think. On the other hand, you can&#x27;t be _totally blind_ either, I&#x27;ve worked on some web apps that were disasters performance-wise.<p>But in general, while I gotta keep performance in mind all the time (okay you&#x27;re right OP), I don&#x27;t really gotta be _measuring_ all the time. The top 3% usually totally gets it.<p>But, when I worked on an ETL project -- performance performance performance all the way. Dealing with millions of records and some expensive transformations, the difference between an end-to-end taking 6 hours and taking 4 hours (or taking one hour! or less!) is _huge_ to how useful or frustrating the software is. And I had to start at the beginning thinking about how basic architectural choices (that would be hard to change later) effected performance -- or it would have been doomed from the start.<p>Certainly a game engine renderer is also more the latter.<p>But I don&#x27;t know if you need _that_ level of performance focus on every project.

The real problem succinctly: people think that quality and quantity are mutually exclusive. Or to go further, that those are also mutually exclusive with inexpensive. That&#x27;s why the industry is flooded with bugs and low quality, cheap labor. Note, I did not say &quot;junior devs&quot; because I&#x27;ve often found the attributes of a high quality developer are more innate (albeit possibly dormant) than taught. If I can write code twice as quickly, it performs twice as well, and it&#x27;s much more maintainable, I have a real hard time emphasizing with anyone&#x27;s legacy&#x2F;performance woes. It&#x27;s like people forget the word premature in the common optimization quote. It&#x27;s not premature to develop with an optimized mindset because it rarely costs anything more than your more expensive salary. You can have a reasonably optimized mindset without needing empirical proof on all but the most nuanced problems.

A refreshing and sane way to think about performance, in a culture of &quot;performance and optimisation are evil and useless; now let us ship our 12MB webpage please&quot;.

Yes! Very much this. This is a lesson that, for example, Apple learned the hard way with Tiger. They now have dedicated performance teams that look at <i>everything</i> throughout the release cycle.<p>I&#x27;d like to refine the advice given a little bit, an approach I like to call &quot;mature optimization&quot;. What you need to do ahead of time is primarily to make sure your code is <i>optimizable</i>, which is largely an architectural affair. If you&#x27;ve done that, you will be able to (a) identify bottlenecks and (b) do something about them when the time comes.<p>Coming back to the Knuth quote for a second, not only does he go on to stress the importance of optimizing that 3% when found, he also specifies that &quot;We should forget about <i>small</i> efficiencies, say about 97% of the time&quot;. He is speaking specifically about micro-optimizations, those are the ones that we should delay.<p>In fact the entire paper <i>Structured Programming with goto Statements</i>[1] is an ode to optimization in general and micro-optimization in particular. Here is another quote from that same paper:<p>“The conventional wisdom [..] calls for ignoring efficiency in the small; but I believe this is simply an overreaction [..] In established engineering disciplines a 12% improvement, easily obtained, is never considered marginal; and I believe the same viewpoint should prevail in software engineering.&quot;<p>That said, modern hardware is fast. Really fast. And the problems we try to solve with it tend towards the simple (JSON viewers come to mind). You can typically get away with layering several stupid things on top of each other, and the hardware will still bail you out. So most of the performance work I do for clients is removing 3 of the 6 layers of stupid things and they&#x27;re good to go. It&#x27;s rare that I have to go to the metal.<p>Anyway, if you&#x27;re interested in this stuff, I&#x27;ve given talks[2] and written a book[3] about it.<p>[1] <a href="http:&#x2F;&#x2F;sbel.wisc.edu&#x2F;Courses&#x2F;ME964&#x2F;Literature&#x2F;knuthProgramming1974.pdf" rel="nofollow">http:&#x2F;&#x2F;sbel.wisc.edu&#x2F;Courses&#x2F;ME964&#x2F;Literature&#x2F;knuthProgrammi...</a><p>[2] <a href="https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=kHG_zw75SjE&amp;feature=youtu.be" rel="nofollow">https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=kHG_zw75SjE&amp;feature=youtu.be</a><p>[3] <a href="https:&#x2F;&#x2F;www.amazon.com&#x2F;iOS-macOS-Performance-Tuning-Objective-C&#x2F;dp&#x2F;0321842847" rel="nofollow">https:&#x2F;&#x2F;www.amazon.com&#x2F;iOS-macOS-Performance-Tuning-Objectiv...</a>

&quot;Fast is my favourite feature&quot; --Someone, maybe from Google? Not sure.

&quot;Premature optimization is the root of all evil&quot; is an amusing quote with some truth to it, but it&#x27;s brought up as some kind of absolute law these days.<p>I&#x27;ve seen it given as an answer on StackOverflow, even when the question is not &quot;should I optimize this?&quot; but more like &quot;is X faster than Y?&quot;<p>We need to stop parroting these valuable, but not absolute mantras and use common sense.

Both sides have merit. The trick is to find a point in between that works for you. What I tend to do after having to optimize after the fact on numerous projects amounts to:<p><pre><code> - write for clarity with an architecture that doesn&#x27;t greatly impede performance - have good habits, always use datastructures that work well at both small and larger scales whenever readily available (e.g. hash tables, preallocating if size is known) - think longer about larger decisions (e.g. choice of datastore and schema, communication between major parts) - have some plans in mind if performance becomes an issue (e.g. upgrade instance sizes, number of instances) and be aware if you are currently at a limit where there isn&#x27;t a quick throw money at the problem next level - measure and rewrite code only as necessary taking every opportunity to share both why and how with as many team members as feasible</code></pre>

I find that the code that looks slow often isn&#x27;t, and the really slow code is always a surprise.<p>I work on something that uses a lot of immutables with copy-modify operations. They never show up in a profiler as a hot spot. The most surprising hot spot was a default logger configuration setting that we didn&#x27;t need. Other hot spots were file API calls that we didn&#x27;t know we&#x27;re slow.<p>I think what&#x27;s more important is to use common sense in the beginning, and optimize for your budget. Meaning: Know how to use your libraries &#x2F; apis, don&#x27;t pick stupid patterns, and only do as much optimization as you have time for.<p>Sometimes an extra sever or shard is cheaper than an extra programmer, or gets you to market on time. Sometimes no one will notice that your operation takes an extra 20ms. Those are tradeoffs, and if you don&#x27;t understand when to optimize for your budget, you&#x27;ll either ship garbage or never ship at all.

I actually saw this tweet the other day. Amusing how often performance is neglected until it kills something.<p>I have also felt it would be a fun bingo game in a year to see when a famous quote of someone would come up. This Knuth quote would definitely be on there.

Not everyone is building web browser, compiler or even a e-commerce site. Even on commerce-related website, only pages on customer acquisition path and buy path really matter.<p>Most of those pages will bubble up when you do your first profiling session anyway.<p>You can get away with good data structures&#x2F;good sql queries and a little big O analysis almost everywhere.<p>Premature optimization, premature nines of stability, premature architecture and abstraction are as evil as ever. They all distract you from moving forward and shipping.<p>Of course, if your product is BLAS library, database, compiler, web browser, operating system or AAA video game, that does not apply. I mean, for most of us &quot;profile often&quot; is a terrible advise.<p>(edit: spelling, clarifications)

This article reminds me of something I read a couple years back that stuck with me.<p>The 10x dev is the dev that creates 10% the problems other devs create.<p>Thinking ahead is a skill the industry at large unfortunately seems to lack

Good performance comes down to using suitable algorithms - not optimization’s after the fact. Thoughtful algorithm choices are never premature.<p>There are also a lot of times when it doesn’t matter, possibly the majority of the time in some domains. I’m working on a project now where the answer takes a couple of seconds to generate but it isn’t needed for minutes so spending time to make it faster would be a waste of my clients money.

&gt; But if performance work has been neglected for most of the product development cycle, chances are that when you finally fire up the profiler it’ll be a uniformly slow mess with deep systemic issues.<p>Hrmm. In my experience very good programs also have very flat profiles. I don’t think a flat profile is indicative of bad performance culture.

While I think that writing simple code is preferred to writing optimized code given a choice, I just hate writing obviously non-optimal code, it leaves bad taste in my mouth. I&#x27;m trying to find some land in between, even if I&#x27;m sure that those optimization efforts won&#x27;t yield any observable gains.

Something I find helpful is to performance and memory profile your projects on a semi-regular basis and establish a baseline. When things suddenly deviate (esp. memory usage) you catch it early before the problem has time to grow.

Nice article, but my eyes hurt from the colors :) Switching to reader mode in Firefox helps a lot.

Rules of optimization:<p>1. Optimize only if needed.<p>2. Premature optimization is the root of all evil.

Replace optimization with security, good design, or any other important facet of software engineering, and you have the same story.<p>Good software is a multifaceted effort, and great software takes care of the important parts with attention to detail where relevant: great games libraries don&#x27;t add significant overhead to frame time, great filesystem libraries don&#x27;t increase read time, great security libraries don&#x27;t expose users to common usage pitfalls creating less secure environments than had you used nothing at all.<p>It happens to be that optimization gets deprioritized at the expense of other things, where &quot;other things&quot; in this context is some category I fail to pin down because PMs don&#x27;t give a shit about what that other category could be, and instead just care that whatever you&#x27;re working on is shipped to begin with.<p>Great software developers will respect the important parts, and still ship. And yes, it&#x27;s always easier to do this from the start than it is to figure it out later. Many things in life are this way.<p>I have a soft spot for performance, though, so I care about this message. One day hardware will reach spacial, thermal, and economic limits and when that day comes, software engineers will have to give a shit, because too many sure as hell don&#x27;t seem to give a shit now.

the article had interesting points but i decided to stop reading because the yellow wall of text with black background was not super readable.

Rule #9. Optimize webpage colors so it doesn&#x27;t hurt people&#x27;s eyes.

Interesting title but the text was far too small to read on my pixel 2 and wouldn&#x27;t let me pinch-zoom in. Optimization of some other metric perhaps?

I don&#x27;t give a fuck, I tell people to buy faster hardware.

Rules of <i>software</i> (or <i>code</i>) optimization maybe? I clicked on this thinking it was going to be about gradient methods.

It&#x27;s interesting to me that people&#x27;s opinions on this subject are much like politics. You either make optimization a priority, or you don&#x27;t optimize prematurely. You write clever well written and commented imperative code, or clear concise functional code.<p>They&#x27;re two completely different schools of thought and may work well in either scenario. It depends a lot on your background, and your current context what way you are going to write code.