For several years I have worked primarily with performance optimizations in the context of video games (and previously in the context of surgical simulation). This differs subtly from optimization in certain other areas, so I figured I'd add my own perspective to this already excellent comment section.<p>1. First and foremost: measure early, measure often. It's been said so often and it still needs repeating. In fact, the more you know about performance the easier it can be to fall into the trap of not measuring enough. Measuring will show exactly where you need to focus your efforts. It will also tell you without question whether your work has actually lead to an improvement, and to what degree.<p>2. The easiest way to make things go faster is to do less work. Use a more efficient algorithm, refactor code to eliminate unnecessary operations, move repeated work outside of loops. There are many flavours, but very often the biggest performance boosts are gained by simply solving the same problem through fewer instructions.<p>3. Understand the performance characteristics of your system. Is your application CPU bound, GPU compute bound, memory bound? If you don't know this you could make the code ten times as fast without gaining a single ms because the system is still stuck waiting for a memory transfer. On the flip side, if you know your system is busy waiting for memory, perhaps you can move computations to this spot to leverage this free work? This is particularly important in shader optimizations (latency hiding).<p>4. Solve a different problem! You can very often optimize your program by redefining your problem. Perhaps you are using the optimal algorithm for the problem as defined. But what does the end user really need? Often there are very similar but much easier problems which are equivalent for all practical purposes. Sometimes because the complexity lies in special cases which can be avoided or because there's a cheap approximation which gives sufficient accuracy. This happens especially often in graphics programming where the end goal is often to give an <i>impression</i> that you've calculated something.
Performance optimization covers a lot of topics, it depends on what you are trying to optimize.<p>1. Latency vs throughput. Oftentimes they are the same, i.e. reduce the time it takes to do something. However, when you passed a certain threshold, techniques that can optimize throughput will hurt latency, so it is important to know what you are looking for. There are also low level details if you have rather extreme latency requirement, e.g. pinning the cores, kernel settings etc.<p>2. Knowledge about the overall system <i>and your input distribution</i>. While this seems trivial, often times you can get large performance improvement by avoiding redundant work, either by caching or lazy evaluation. Some computation may only exist because they <i>may</i> be needed later, and these can be avoided by lazy evaluation.<p>3. Better algorithms. Again, this seems trivial but oftentimes people are using algorithms that are far from optimal. And even if the algorithm can be asymptotically, there may be faster algorithms for special cases or faster in practice. Optimizing special cases may be rewarding if they occur frequently. Do you really need optimal solutions? Can you allow randomization? Can you do optimization on the queries to make it faster <i>overall</i> without optimizing individual operations?<p>4. Parallelization. Can you do parallelization? Are your problem instances large enough, or individual stages slow enough to benefit from parallelization? Do you have computation that are trivially parallelizable and can benefit from offloading to the GPU? If your code is waiting on some events, can you make them async? Can you avoid locks or atomic operations in your parallel code?<p>5. Data structure optimization. Can you reduce the number of allocation needed? Can you make the data structure more linear and predictable so the CPU can have better cache utilization? Can you compress certain data if they are sparse?<p>6. Low level CPU/GPU optimizations. There are a lot of great resources out there, but only do it when you are very sure it will be worth it, i.e. they are bottleneck in your system.
Former HFT dev here. Know fundamentals: sources of performance issues = things that eat/waste CPU cycles, things that reach too far down the memory hierarchy. Usually the latter. E.g. L2 cache to RAM - order of magnitude slower; RAM to disk: 4+ orders of magnitude slower.<p>Things that eat CPU: iterations, string operations. Things that waste CPU: lock contentions in multi-threaded environments, wait states.<p>You can usually build a lot of the understanding from first principles starting there. Back in the day we had to do this because there wasn't much by way of readily available literature on the subject. Actual techniques will depend or evolve based on your choice of platform or version.<p>E.g. 20 years ago, we used to create object pools in C++ at load time to avoid Unix heap locks at runtime. This may no longer be necessary. 15(ish?) years ago, JNI was used when the JVM wasn't fast enough for certain stuff. This is no longer necessary. 10 years ago, immutable JS objects were thought to be faster because the JS runtimes at the time were slower to mutate existing objects than to create new ones. This too, may no longer be true (I haven't checked recently). Until very recently, re-rendering with virtual DOM diffing was considered more performant than direct, incremental DOM manipulation. This too, may no longer be true.
Casey Muratori did many lectures. Check out his yt page playlist. Start with the one titled software quality
<a href="https://www.youtube.com/@MollyRocket/playlists" rel="nofollow">https://www.youtube.com/@MollyRocket/playlists</a><p>I heard good things about his course <a href="https://www.computerenhance.com/" rel="nofollow">https://www.computerenhance.com/</a><p>Agner Fog manuals are good too <a href="https://www.agner.org/optimize/#manuals" rel="nofollow">https://www.agner.org/optimize/#manuals</a><p>A site to look up instruction timings is <a href="https://uops.info/table.html" rel="nofollow">https://uops.info/table.html</a>
If you want to learn how to understand the performance of the whole system I can recommend Brendan Gregg's Systems Performance: Enterprise and the Cloud (<a href="https://www.brendangregg.com/blog/2020-07-15/systems-performance-2nd-edition.html" rel="nofollow">https://www.brendangregg.com/blog/2020-07-15/systems-perform...</a>). It is a good book that teaches a lot of basics and techniques and gives a good understanding of the impact different system components can have on performance.
Please try to remember that one of the the most abused quotes in Software Engineering is the old Knuth chestnut about "Premature optimization is the root of all evil."<p>The full quote is as follows, "Programmers waste enormous amounts of time thinking about, or worrying about, the speed of noncritical parts of their programs, and these attempts at efficiency actually have a strong negative impact when debugging and maintenance are considered. We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil. Yet we should not pass up our opportunities in that critical 3%."<p>He specifically does not recommend writing code that is obviously inefficient. He is clearly referring to engineers optimizing routines by introducing new, additional complexity. He is not recommending that anyone write obviously, ruinously slow, bloated code.<p>Writing software is an art and a science.<p>Optimization is no different. One frequently missing part of our process is to keep a watchful eye on features that are obviously toxic to performance during development (i.e., multiply-nested for loops, many large external dependencies, introducing and frequently iterating over huge, bloated structs, etc.).<p>As everyone says, measure early and often, but also please don't just shout "LEEEEEEEROY JENKINS!" as you throw fireballs of slow, bloated code into the world.
Denis Bakhvalov has some great resources for this:<p>1. His free course: <a href="https://products.easyperf.net/perf-ninja" rel="nofollow">https://products.easyperf.net/perf-ninja</a><p>2. His free book: <a href="https://book.easyperf.net/perf_book" rel="nofollow">https://book.easyperf.net/perf_book</a> (the 2nd edition is being worked on right now and there's a draft on github: <a href="https://github.com/dendibakh/perf-book">https://github.com/dendibakh/perf-book</a>)
Not a comprehensive set of resources as you asked, but I want to share one line of thought that had a profound impact in my way of working.<p>Think of a system as a chain of bottlenecks, visualized as a set of pipes. If you can measure the metric you care about (tput, latency etc) at a component level, and put together the system’s control flow, you can spot where the bottleneck is. Optimise that component, and you will reveal the next bottleneck, now optimize that… and it goes on. To limit the fun of this exercise, it helps to do a back of the envelope calculation of what is a realistic estimate of the thing you measure in the system. Example - I want this service to do 100 emails/ sec. Now, piece by piece remove bottlenecks to achieve close that value.
If you need to organize your thoughts on what measurements are and how at least some profiling tools work you can pick up a book or two. I would recommend for example [1]. It is a bit heavy on C++ side but you can complement it with something relevant to your job's language.<p>If you want one bit of advice on optimization, I can try one: follow your app architecture closely. This is where data structures that hold all of the important data live and this is what limits what is possible to achieve on performance. A lot of learning is narrowly focused to specific micro optimization techniques leaving big picture as an exercise.<p>1 Fedor Pikus, The Art of Writing Efficient Programs
<a href="https://en.algorithmica.org/hpc/" rel="nofollow">https://en.algorithmica.org/hpc/</a><p>This is a good book. It covers most common concepts and techniques in a fairly accessible way. At they end it also shows builds up a highly optimized version of some algorithms and data structures and does explains every optimization.
Performance optimization is very simple. Make computer do less to get same or similar result. The most performant application does very little and still gets you the result you need.<p>To do this you must find ways to “cheat”. This can be of various forms. Better algorithms, better data structures, precomputation, caching. At some point you will exhaust low hanging fruit and need to dig into lower level aspects of the code or its compilation.<p>Anyway, best way to learn is to do it, go depth first and always check your work thoroughly. (It is easy to optimize yourself into a solution that is not working properly).
Understand first, then fix. And you understand by measuring the right thing at the right time (scope). Systemwide resource utilization averages are not gonna tell you where your critical thread or database connection is spending their time at - you need to measure (profile) precisely where your task of interest is spending their time.<p>I've learned a lot from Cary Millsap over the last 2 decades and he recently published a general performance optimization book "How to Make Things Faster" that I can recommend [1]. It's less about tools, more about the method and systematic approach for performance optimization:<p>[1] <a href="https://method-r.com/books/faster/" rel="nofollow">https://method-r.com/books/faster/</a>
work with steve jobs, or someone like him.<p>> One of the best, if possibly exaggerated, examples of the reality distortion field comes from Jobs's biographer Isaacson. During development of the Macintosh computer in 1984, Jobs asked Larry Kenyon, an engineer, to reduce the Mac boot time by 10 seconds. When Kenyon replied that it was not possible to reduce the time, Jobs asked him, "If it would save a person's life, could you find a way to shave 10 seconds off the boot time?" Kenyon said that he could. Jobs went to a white board and pointed out that if 5 million people wasted an additional 10 seconds booting the computer, the sum time of all users would be equivalent to 100 human lifetimes every year. A few weeks later Kenyon returned with rewritten code that booted 28 seconds faster than before.<p><a href="https://en.m.wikipedia.org/wiki/Reality_distortion_field" rel="nofollow">https://en.m.wikipedia.org/wiki/Reality_distortion_field</a>
Google publishes some of its data center optimization lessons and tips at <a href="http://abseil.io/fast" rel="nofollow">http://abseil.io/fast</a>. This includes topics like higher-level methodology and goal setting, these topics are often less covered by other resources.<p>Full disclosure: I'm the editor in chief for the series.
> This is an area that is new to me, but a big part of my new job.<p>Can you tell me more about what your new job is, without releasing anything sensitive?<p>If you are running applications on Linux containers in the cloud, then I would recommend Brendan Gregg's blog and books (<a href="https://www.brendangregg.com/overview.html" rel="nofollow">https://www.brendangregg.com/overview.html</a>). He does a lot of knowledge sharing from his experiences at Netflix.
What you really want to learn about is observability, benchmarking and instrumentation. Once you are an expert in these topics for your domain, optimization will be about making obvious choices within localized constraints.
Most software in the industry is slow because it's doing a lot of stuff that it shouldn't. Often times additional "optimization" layers adds caching, but makes getting to the root of the issue harder. The biggest win is primarily getting rid of things you don't need and secondarily operating on things in batch.<p>My playbook for optimizing in the real world is something like this:
1. Understand what you're actually trying to compute end-to-end. The bigger the chunk you're trying to optimize, the greater the potential for performance.<p>2. Sketch out what an optimal process would look like. What data do you need to fetch, what computation do you need to do on this, how often does this need to happen. Don't try to be clever and micro-optimize or cache computations. Just focus on only doing the things you need to do in a simple way. Use arrays a lot.<p>3. Understand what the current code is actually doing. How close to the sketch above are you? Are you doing a lot of I/O in the middle of the computation? Do you keep coming back to the same data?<p>If you want to understand the limits of how fast computers are, and what optimal performance looks like I'd recommend two talks that come with a very different perspective from what you usually hear:<p>1. Mike Acton's talk at cppcon 2014
<a href="https://www.youtube.com/watch?v=rX0ItVEVjHc" rel="nofollow">https://www.youtube.com/watch?v=rX0ItVEVjHc</a><p>2. Casey Muratori's talk about optimizing a grass planting algorithm
<a href="https://www.youtube.com/watch?v=Ge3aKEmZcqY" rel="nofollow">https://www.youtube.com/watch?v=Ge3aKEmZcqY</a>
If you're new to this area, I would first start by understanding which profiling tools you can use depending on the OS, languages and systems involved.<p>Even if your system is not C++, I've always enjoyed this talk and the subsequent discussion which tackles some of the problems associated with some programming practices and the impact on performance.<p>CppCon 2014: Mike Acton 'Data-Oriented Design and C++'
<a href="https://youtu.be/rX0ItVEVjHc" rel="nofollow">https://youtu.be/rX0ItVEVjHc</a>
I really liked listening to and working on the projects of the open MIT course about performance
<a href="https://ocw.mit.edu/courses/6-172-performance-engineering-of-software-systems-fall-2018/pages/syllabus/" rel="nofollow">https://ocw.mit.edu/courses/6-172-performance-engineering-of...</a>
Write a piece of software in a week - a full app, with discrete functionality that would challenge you to deliver on time. Do it, and burn through it.<p>Then optimize it - measure front end render performance/compilation times/code perf, and then do the same on the backend. Write a blog post about it.<p>No substitute for experience
The book Understanding Software Dynamics by Richard Sites is all about performance optimization<p><a href="https://www.amazon.in/Understanding-Software-Addison-Wesley-Professional-Computing/dp/0137589735" rel="nofollow">https://www.amazon.in/Understanding-Software-Addison-Wesley-...</a>
You’re unlikely to find a good answer because it’s a very specialized skill that is mostly from experience doing it, and HN tends to self select out of that pursuit.
I've been working in the performance industry (focusing on front-end performance and UX) for 15 years. Some resources you might find helpful:<p>• Steve Souders' books 'High Performance Web Sites' & 'Even Faster Web Sites' – <a href="https://stevesouders.com/" rel="nofollow">https://stevesouders.com/</a> – Steve literally wrote the book(s) on performance and is widely considered the godfather of the industry.<p>• WPO stats – <a href="https://wpostats.com/" rel="nofollow">https://wpostats.com/</a> – Case studies and experiments demonstrating the impact of performance optimization on UX and business metrics<p>• performance.now() talks – <a href="https://www.youtube.com/c/WebConferencesAmsterdam/videos" rel="nofollow">https://www.youtube.com/c/WebConferencesAmsterdam/videos</a> – PerfNow is the only annual global conference dedicated to performance. You can watch past talks on their YouTube channel.<p>• Intro to Web Performance – <a href="https://support.speedcurve.com/docs/psychology-of-web-performance" rel="nofollow">https://support.speedcurve.com/docs/psychology-of-web-perfor...</a> – This is a collection of introductory articles covering topics like the psychology of sit speed to Core Web Vitals to how to create a performance culture in your organization.
There is a MIT course on YouTube and also, there is a pretty famous former M$ performance engineer who worked on Xbox and bunch of other large projects, he has webpage about how he tracks down bugs and performance issues, don't it have it handy unfortunately.
Another thing to look at - low level optimization. There is a cool book, two volumes written by a German guy - I don't have a link for it either. Maybe someone who has those links can post them here.
EDIT:
<a href="https://www.agner.org/optimize/" rel="nofollow">https://www.agner.org/optimize/</a>
Suggestion: Program some (slow) microcontrollers as a hobby.<p>Go multi-core because async is an important optimization skillset, but other than that just build some things.<p>I live and breathe optimizations (it feels almost as satisfying to me as driving fast) and as an example recently I created an 11-board (one for each channel) wifi-presence-detection system in a busy wifi area and there was literally no way it was going to work without optimization. From communication protocol to having to be strict about every byte of memory, it’s working with the first principles that built the entire industry.
For me, one of the biggest leaps in how I think about performance was when I learned about the Top-Down Micro-Architecture Analysis Method, by Ahmad Yasin from Intel. You can learn the main ideas from himself in the video below:<p><a href="https://youtu.be/kjufVhyuV_A" rel="nofollow">https://youtu.be/kjufVhyuV_A</a><p>The idea to classify cycles into front-end bound, backend bound, bad speculation or memory bound is brilliant. Once you know which one your program suffers from, it's easy to know what can be done to improve things.
Measure everything and be extremely critical. Be ready to challenge common and popular held assumptions.<p>Here is something I wrote about extreme performance in JavaScript that is discarded by most programmers because most people that program JavaScript professionally cannot really program.<p><a href="https://github.com/prettydiff/wisdom/blob/master/performance_frontend.md">https://github.com/prettydiff/wisdom/blob/master/performance...</a>
One thing to keep in mind is there's three layers of optimisation:<p>1. The problem,
2. The algorithms,
3. Micro-optimisation.<p>The potential gains shrink rapidly as you descend this list. A lot of people start thinking at level 3 straight away, but this is pointless if you've left performance on the table at the higher levels. For example, no amount of clever bit twiddling will compensate for the wrong algorithm, and even the best algorithm is pointless if you're solving the wrong problem.
Michael Abrash's stuff is still worth reading:<p><a href="https://news.ycombinator.com/item?id=20883860">https://news.ycombinator.com/item?id=20883860</a>
I enjoyed talks about optimization by Casey Muratori. Some examples:<p><a href="https://m.youtube.com/watch?v=Ge3aKEmZcqY" rel="nofollow">https://m.youtube.com/watch?v=Ge3aKEmZcqY</a><p><a href="https://m.youtube.com/watch?v=ffDXc6oup3Q" rel="nofollow">https://m.youtube.com/watch?v=ffDXc6oup3Q</a><p><a href="https://m.youtube.com/watch?v=pgoetgxecw8" rel="nofollow">https://m.youtube.com/watch?v=pgoetgxecw8</a>
What kind of work are you doing? There are some shared ideas (measure, do less work, etc.) but the best advice would probably be tailored to what you’re working on.
I think the Optimization course by Prof. Jacco is a nice start <a href="https://web.archive.org/web/20230924064410/https://www.cs.uu.nl/docs/vakken/mov/" rel="nofollow">https://web.archive.org/web/20230924064410/https://www.cs.uu...</a> (sadly the website seems to be currently down). Basically you need a mental framework to approach optimizing software (else you might just be spinning around wasting time). I recommend reading at least lecture 1 and looking at the references on the website.<p>As for specific optimizations, it requires context of what software are you trying to optimize and under what circumstances. A lot of the times you are going to see that the answer to asking if certain optimizations are worth the effort is going to be 'it depends'
Understand how the software works, and how computers work in general. You have to understand the system before you can a) understand how it's slow, and b) how to make it faster. If you can tell us what, specifically, you need to optimize, we can recommend more specific techniques.<p>Or get a job you can handle idk.
Here's 2 more links. I didn't see already posted worth watching/reading. Should give a good intro. Within 3 hours combined (For CPU performance anyway).<p>A good talk, doesn't go deep and instead goes a bit wide.
<a href="https://www.youtube.com/watch?v=6RlloT_6WxA" rel="nofollow">https://www.youtube.com/watch?v=6RlloT_6WxA</a><p>This one explains the black magic the CPU makes have been doing. If you going to be optimizing code you should know your hardware.
<a href="https://www.lighterra.com/papers/modernmicroprocessors/" rel="nofollow">https://www.lighterra.com/papers/modernmicroprocessors/</a><p>Aditional note: I've noticed C++ conventions have a habit of having performance related talks, YT is your friend.
Has you mention "new job", and everyone is talking you about computers... I will mention the other side:<p>1) Try to understand well the architecture of your company (who is who, who decides what changes are made to computers, how they decide that, what metrics do they use, what tests and benchmarks are passed before changes, etc)<p>2) Try to understand your place in such architecture. Am I responsible from the overall performance? or only the performance of certain components? am I responsible about the latency of the network or the latency of the database, or both etc. Make a clear scope. This will help you to focus on which metrics do you need to follow.<p>3) Try to understand the company procedures. Can I refuse a change that comes from the product or marketing team? can I refuse a change that comes from developers? can I refuse a change that comes from the platform team? how much time do I have to analyze such changes before they reach production, how can I request the rollback of a unsupervised change, where can I check the performance impact of each change made on production in the past? etc<p>4) Try to understand what the CEO and CTO, your team and the rest of teams should expect from you. Are there any SLA o SLO for your position related to the overall performance?<p>5) Make clear how you are informed of ongoing changes and roadmaps? Should I spend all the week in the performance of a component that is going to be deprecated in the next sprint? etc<p>6) Ask doubts and questions to your team mates, or department head. They may teach you about the company workflows, past issues, past solutions, corner cases, blockers, resources, plans and guidelines.<p>In short... look for reading/watching material, but don't forget to look at your company too, you will find things to learn there too, and maybe things that need to change if they are important enough; or need to be clarified that they are not important enough to change, to defend your work on future performance issues, related to those things that weren't changed.
The most interesting thing I've learned in this regard (from Casey Muratori) is non-pessimization. Non pessimization means don't make the computer do unnecessary work. Just write the simplest code that does the thing. Unfortunately almost no software is written like that.
To be honest I think the best way to learn about it is to develope for resource constrained environments. E.g. when you use 99% of your embedded MCUs code memory and another static string for a label shown on screen stops your code from compiling you <i>will</i> optimize code.
Well, start with literature on the specific domain of your new job. That way, you can learn what "performance" even means on your area, what are the common problems, what to measure, and what kind of knowledge you need.
The book "Writing Efficient Programs", by Jon Bentley, is still a valuable resource. See the short subthread starting with a comment I posted here some years ago:<p><a href="https://news.ycombinator.com/item?id=13407192">https://news.ycombinator.com/item?id=13407192</a><p>A few people had replied, agreeing with my opinion, and giving some more details. One of them called the book "gold".<p>I have also posted about the book a few other times on HN, over the years.<p>Those comments can be found by searching hn.algia.com for comments (not stories) matching the pattern "writing efficient programs vram22".
Plugging the book that I just published: <a href="https://a.co/d/iTjaQzP" rel="nofollow">https://a.co/d/iTjaQzP</a><p>It's a beginner-friendly introduction to Low Latency Programming, which involves a lot of performance optimization. Could be a good way to start your learning on the subject.<p>You can read one of the chapters on my blog:
<a href="https://tech.davidgorski.ca/introduction-to-low-latency-programming-minimize-branching-and-jumping/" rel="nofollow">https://tech.davidgorski.ca/introduction-to-low-latency-prog...</a>
Huge topic, what are you trying to optimize? What language(s), hardware, etc.<p>Optimizing games sends you deep down a fun rabbit hole, but that will be very different than trying to optimize a Go backend server.
The Nike doctrine works: just do it. Besides that I recommend always ask yourself the question: we told the computer to do X, and it took too long, so what was it doing for that time? The rough answer can come from surprisingly simple sources such as "top". Fancy, intrusive tools such as traditional profilers are often not the best first place to look for answers. If X is some short one-off thing that makes it hard to see what's happening: make it do 1M of X so bulk data can be observed.
Agner Fog’s optimization manuals are pretty good
<a href="https://www.agner.org/optimize/" rel="nofollow">https://www.agner.org/optimize/</a>
Many have covered specific learning material, but my best advice is to find a mentor. It really is a skill that's best learned by apprenticing. I new a lot of concepts and could muddle my way through them, but it wasn't until I had experienced people to work under directly that my skills really took off.<p>Just like the best advice on learning to write code is to write code, the best way to learn how to optimize performance is to optimize performance.
For a specific introduction to database optimization, I can recommend Silvia Botros and Jeremy Tinley: "High Performance MySQL" -- <a href="https://www.oreilly.com/library/view/high-performance-mysql/9781492080503/" rel="nofollow">https://www.oreilly.com/library/view/high-performance-mysql/...</a>
There's <i>The Every Computer Performance Book</i> <a href="http://www.treewhimsy.com/TECPB/Book.html" rel="nofollow">http://www.treewhimsy.com/TECPB/Book.html</a> and the blog it's based on, <a href="https://rwwescott.wordpress.com/" rel="nofollow">https://rwwescott.wordpress.com/</a>
This has been very helpful to me: <a href="https://viralinstruction.com/posts/hardware/#74a3ddb4-8af1-11eb-186e-4d80402adfcf" rel="nofollow">https://viralinstruction.com/posts/hardware/#74a3ddb4-8af1-1...</a><p>It’s really not specific to Julia, though the language does let you drill down into the details nicely.
>What are some good resources for learning about performance optimization?<p>I swear, nobody actually read OP's post. The top <i>five</i> comments are "here's some personal advice about general rules of thumb, without any links to actual resources!"<p>Kudos to levodelellis, whose post is at #6 root comment and contain some links and drops some names.
I think this thread is old enough to ask something like this, but instead of learning performance optimizatoin, is there a way to be hired to work on this? I am fascinated and I always loved when I had to optimize something (mostly code in my experience, algorithms, etc), but that's only a very small percentage of the work I do.
The difficult thing is to benchmark the software correctly and evaluate the impact of a change. Most of the examples on the Internet are useless micro-optimizations. I evaluated a program some time ago that did several speed tricks. But the reason it was slow was because it reread a file on each iteration in a loop.
If you are looking at the .Net ecosystem, I can't recommend this book enough. The chapter on Garbage Collection itself was worth the price of the book to me:
<a href="https://www.writinghighperf.net/" rel="nofollow">https://www.writinghighperf.net/</a>
1. Computer Enhance by Casey Muratori (check out some of his YouTube videos if you want a preview)<p>2. Read the blog posts people wrote about the 1BR challenge where they tried to figure out the fastest way to process 1 billion rows of data<p>3. Brendan Gregg‘s blog<p>4. Google/YouTube how to profile code in your desired language
Read some of the performance-related work here: <a href="https://acl.inf.ethz.ch/publications/" rel="nofollow">https://acl.inf.ethz.ch/publications/</a>
If you are in “javaland” look at billion row challenge, you will learn a lot - <a href="https://github.com/gunnarmorling/1brc">https://github.com/gunnarmorling/1brc</a>
Mature Optimization by Carlos Bueno
<a href="https://carlos.bueno.org/optimization/" rel="nofollow">https://carlos.bueno.org/optimization/</a>
It really depends on what level you're working on improving. It's effectively queues all the way down, but programmers hate reading statistics. E.g. a server process is a series of queues between your TCP socket to your process to your disk, CPU's reorder buffer, and scheduler.<p>You have three areas to study:<p>1. Measurement - makes you define the performance you're looking for and measure it. Until you do this it's mostly a bullshit "make people stop complaining about performance" errand that's too wishy washy to do with more than a few stabs in the dark. With containers and decent capture of samples of your load, a benchmark is pretty straightforward to set up.<p>2. Modeling - these models are usually little more than measured rates and latencies applied to Little's Law. Pocket-calculator math is often good enough. At worst, an M/M/1 queue.<p>3. Instrumentation - Figuring out how to attribute your computer's resources (memory, CPU time, iops, etc) to different parts of your code. Tracing libraries, Linux perf, and ebpf can be useful here.<p>There are a decent number of computers performance books. I like the ones by Jain (great, but AFAICT out of print) and Harchol-Baltar. For work, you shouldn't read them straight through but iterate through parts as you better understand the problem you're trying to solve and start choosing strategies. For the tactical side. Brendon Gregg (sp?) has some decent measurement tool books. Figure out what you want to improve and how to measure that. Then start attributing the existing performance to implementation choices that you can control. Then control those choices (e.g. change algorithm, load balance better, make design trade-offs) to improve performance.
Watch this video <a href="https://www.youtube.com/watch?v=4LiP39gJuqE" rel="nofollow">https://www.youtube.com/watch?v=4LiP39gJuqE</a>
I'd recommend learning how to instrument and measure the performance of your code. I find most performance issues are (mostly) situations you didn't and couldn't anticipate. So instead of preventing them, learning to investigate and fix them is key. (Shout out to Brendan and his Linux Performance page <a href="https://www.brendangregg.com/linuxperf.html" rel="nofollow">https://www.brendangregg.com/linuxperf.html</a>).<p>Second there is an important engineering lesson to learn. Often there are many performance issues, with only a few acting as serious bottlenecks. Additionally sometimes the solutions to performance issues add complexity, but as an engineer you want to avoid complexity. Engineering effort is usually limited, so there is always a question of whether a performance issue needs to be fixed now or left till later.<p>Here is a quick example to illustrate my point. pgAdmin is a webui program to interact with PostgreSQL databases, allowing you to remotely run queries. Part of its operations fetches information about columns in a result set, in one version this code ran one query per column sequentially. So c columns, each a synchronous query to the server - almost instant on a local database with a small number of columns. However with 400 columns, and a 40ms internet link, it ended up taking at least 400*40=16 seconds to complete. In 99% of cases this code works just fine, but in a few less obvious scenarios its runtime balloons.<p>Another example; what happens if all the daily scheduled jobs run at the same time? <a href="https://github.com/go-acme/lego/issues/1656">https://github.com/go-acme/lego/issues/1656</a>
As you can tell from the diversity of responses here it really depends on what you're doing. In my work I use C++, and "optimization" typically involves making a heavy computation run faster (measured in wall clock time) or making a particular subsystem use less memory.<p>The number one most important thing you can do is dive in and start profiling real-world code. Find a part of your software that is too slow or uses too many resources, and use whatever the standard profiler is for your development environment to figure out why. Performance optimization is a very empirical discipline. Yes there are general principles, but if you don't measure your baseline or your changes you won't know how good your optimization was. In my experience, the first attempt at a fix is often flat-out wrong! Doing this first will also help motivate your reading.<p>Once you know how to measure the performance of your software, I recommend learning the basics of modern computer architecture. At a minimum, learn about CPU caches, how they work, and how to design your code to use them effectively. I find Algorithms for Modern Hardware to be a good resource for this [1], but there are many others. Relatedly, you should have a rough idea of how long it takes for your computer to do various basic things (fetch something from memory, fetch something from cache, etc.). There's a table at [2] that gives a good idea. Don't worry too much about the absolute values--the order of magnitude is what's important.<p>You should also study fundamental data structures, but understand that for low-level programming 95% of the time the correct answer will be to shove everything into a simple flat array (e.g. std::vector in C++), maybe with some sort of index on top. Fancy data structures are more important in higher-level languages that are structurally unable to make effective use of modern hardware.<p>[1] <a href="https://en.algorithmica.org/hpc/" rel="nofollow">https://en.algorithmica.org/hpc/</a><p>[2] <a href="https://gist.github.com/jboner/2841832" rel="nofollow">https://gist.github.com/jboner/2841832</a>
It really depends on where you sit in the stack.<p>The generally useful rule is “measure before acting”.<p>There are some rules of thumb at every layer:<p>If you’re getting bad scrolling in a web application on a mobile phone, something is probably getting called over and over<p>If you’ve got an x86_64 server maxing out but the cores aren’t printing work? Zen4’s northbridge has some edge cases.<p>If you’re trying to melt aluminum so that exquisite optics can do extreme ultra-violet litho: weak hyper charge is very well determined empirically but there are some weird readings on muon spin.<p>I’m sort of kidding because this is an Endless Internet Feud, but really it’s measure and whack the hot spots.<p>I’ve done a bunch of this shit: if you’re not sure where to start feel free to email.
I admire Daniel Lemire’s work on SIMD implementations. [Lemire]<p>[Lemire] <a href="https://lemire.me/en/#publications" rel="nofollow">https://lemire.me/en/#publications</a><p>I learn a lot by reading my compiler’s and profiler’s documentation.<p>For Rust, the <i>Rust Performance Book</i> by Nicholas Nethercote et al. [Nethercote] seems like a nice place to start after reading the Cargo and rustc books.<p>[Nethercote] <a href="https://nnethercote.github.io/perf-book/" rel="nofollow">https://nnethercote.github.io/perf-book/</a><p><i>Algorithms for Modern Hardware</i> by Sergey Slotin [Slotin] is a dense and approachable overview.<p>[Slotin] <a href="https://en.algorithmica.org/hpc/" rel="nofollow">https://en.algorithmica.org/hpc/</a><p>Quantitative understanding of the underlying implementations and computer architecture has been invaluable for me. <i>Computer architecture: a quantitative approach</i> by John L. Hennessy and David A. Patterson [H&P] and <i>Computer organization and design: the hardware/software interface</i> by Patterson and Hennessy [P&H ARM, P&H RISC] are two introductory books I like the best. There are three editions of the second book: the ARM, MIPS and RISC-V editions.<p>[H&P] <a href="https://www.google.com/books/edition/_/cM8mDwAAQBAJ" rel="nofollow">https://www.google.com/books/edition/_/cM8mDwAAQBAJ</a><p>[P&H ARM] <a href="https://www.google.com/books/edition/_/jxHajgEACAAJ" rel="nofollow">https://www.google.com/books/edition/_/jxHajgEACAAJ</a><p>[P&H RISC] <a href="https://www.google.com/books/edition/_/e8DvDwAAQBAJ" rel="nofollow">https://www.google.com/books/edition/_/e8DvDwAAQBAJ</a><p>Compiler Explorer by Matt Godbolt [Godbolt] can help better understand what code a compiler generates under different circumstances.<p>[Godbolt] <a href="https://godbolt.org" rel="nofollow">https://godbolt.org</a><p>The official CPU architecture manuals from CPU vendors are surprisingly readable and information-rich. I only read the fragments that I need or that I am interested in and move on. Here is the Intel’s one [Intel]. I use the Combined Volume Set, which is a huge PDF comprising all the ten volumes. It is easier to search in when it’s all in one file. I can open several copies on different pages to make navigation easier.<p>Intel also has a whole optimization reference manual [Intel] (scroll down, it’s all on the same page). The manual helps understand what exactly the CPU is doing.<p>[Intel] <a href="https://www.intel.com/content/www/us/en/developer/articles/technical/intel-sdm.html" rel="nofollow">https://www.intel.com/content/www/us/en/developer/articles/t...</a><p>Personally, I believe in automated benchmarks that measure end-to-end what is actually important and notify you when a change impacts performance for the worse.