Optimizing performance
Better Code, Better Science: Chapter 10, Part 1
This is a possible section from the open-source living textbook Better Code, Better Science, which is being released in sections on Substack. The entire book can be accessed here and the Github repository is here. This material is released under CC-BY-NC-ND.
In the next set of posts I will turn to the issue of when and how to optimize the performance of our code.
The computing power available to scientists has increased in a shockingly consistent way since the first microprocessors were manufactured in the 1970’s. The left panel in Figure 10.1 shows that the number of transistors on commercial CPU chips has doubled about every two years since the 1970s, very close to the two year doubling time predicted by Gordon Moore in 1975 (Moore et al. 1975). The number of transistors relates only indirectly to the computing power of the machine, and the right panel in Figure 10.1 shows that the computer power of the world’s top supercomputer (measured in the number of floating point operations per second) has increased even faster, doubling roughly every 1.2 years.
Figure 10.1: A plot of increased computing power over time. Left panel shows transistor count in commercially available microprocessors over time on a log scaled axis, based on data from en.wikipedia.org/wiki/Transistor_count. The plot shows a consistent logarithmic increase in transistor count, with an estimated doubling time of about 2.16 years. Right panel shows performance of the world’s top supercomputer (in GigaFLOPS - billion floating point operations per second) on a log scaled axis, showing an even faster doubling time of about 1.2 years, based on data obtained from en.wikipedia.org/wiki/History_of_supercomputing.
To put this into a personal perspective, I published my first paper using computer simulations in 1996 (Poldrack 1996), based on simulations I had performed in 1994-5 as a graduate student. I ran my simulations on a Power Macintosh, and I remember them taking many hours to complete, but let’s say that I had access to the top supercomputer of the day back in 1994. The top supercomputer in 2025 was more than ten million times faster than the top machine in 1994. If simulation time scaled directly with processor speed, this would mean that a simulation that takes one minute on the latest machine would have taken over 19 years in 1994! This kind of scaling is not generally true; as we will discuss below and in the next chapter on high-performance computing, there are many factors beyond CPU speed that can limit the speed of computing operations. Further, taking advantage of these new systems requires the use of parallel processing, which often requires substantial changes in software architecture.
I raise these comparisons to highlight the fact that any attempts to optimize the performance of our code will often be much less effective than simply finding a more powerful computer. However, it’s often the case that small changes in our code can have significant performance impacts. In this chapter I will highlight the ways in which one can judiciously optimize the performance of code without significantly impacting the quality of the code.
10.1 Avoiding premature optimization
Donald Knuth, one of the founders of the modern field of computer science, is famous for saying the following about code optimization (Knuth 1974):
The real problem is that programmers have spent far too much time worrying about efficiency in the wrong places and at the wrong times; premature optimization is the root of all evil (or at least most of it) in programming.
I think that it’s important to know how to optimize code, but also important to know when to optimize code and when not do do so. In their book The Elements of Programming Style, Kernighan & Plauger (1978) proposed a set of organizing principles for optimization of existing code that highlight the tradeoffs involved in optimization:
“Make it right before you make it faster”
“Make it clear before you make it faster”
“Keep it right when you make it faster”
That is, there is a tradeoff between accuracy, clarity, and speed that one must navigate when optimizing code. I would focus on optimization after you have code that runs on a small example problem, and any of the following occurs:
Something simple seems like it’s taking much longer than it seems like it should
Scaling to larger problems takes exceedingly long and you don’t have access to a larger computer system