I’m looking for resources (ideally a book) that will help me gain an in-depth understanding of C++ performance. Here’s a little more background:
I write server software with very high throughput requirements and/or low latency requirements. We write in C++; it’s not really up for debate at this time. Most of my colleagues seem to have a much better understanding of C++ performance. They have better mental models, so they can tell when a certain bit of code will perform poorly at scale. I lack this understanding, and so I’m looking to improve my mental model.
I am specifically interested in:
- Understanding cache effects, and how cache locality due to object layout affects my code’s performance. This is the number one issue that seems to be brought up by other members of my team.
- Understanding how memory allocation otherwise affects performance. Should I be using TCMalloc (or other mallocs), and how should I know? How should I tune the various allocation and deallocation parameters?
- How do I know when the overhead from object copying will matter (and therefore should switch to pointers, for example)?
- I’m also generally interested in “optimizations” as long as I have the knowledge as to when to use them.
Things I’m not really interested in:
- “High Performance Computing,” a term which seems indicate more math/simulation-oriented applications.
- Discussions of C++ performance relative to other languages, since I’m stuck with C++.
As a starting point, anyone know if this book, Efficient C++, would fit the bill?
Let me split my recommendations into several sections.
C++ Optimization
As a starting point, I would highly recommend Agner Fog’s Optimizing software in C++. This manual gives an excellent overview of the common C++ optimization topics.
Understanding Hardware in General
To have a good mental model of the C++ performance, you also need to understand the underlying hardware. Consider this statement:
On the C++ language side, the line of code is boring from the performance perspective: it’s just one memory write. But, on real hardware, the performance of that memory write can vary by orders of magnitude. To understand what’s going on at that level, you need to learn about concepts such as caches, processor pipeline, TLB, branch prediction, and so on.
As a quick introduction to processor caches, I’d recommend my article Gallery of Processor Cache Effects. A much deeper and longer (>100 pages) discussion of caches and computer memory is What Every Programmer Should Know About Memory.
To get an overall understanding of modern computer hardware, Computer Architecture: A Quantitative Approach is the commonly recommended book. I didn’t read the book myself, but instead learned by reading blogs and experimentation. However, others have apparently found the book to be very useful.
Understanding Specific Processors
At one point in your journey to improve your optimization skills, you’ll find it useful to recognize specifics of different processors. As one example of many, different Intel and AMD processors have very different penalties for using unaligned SSE instructions, such as the _mm_storeu_ps C++ intrinsic.
To learn the specifics of different processors, I’d recommend The microarchitecture of Intel, AMD and VIA CPUs: An optimization guide for assembly programmers and compiler makers. In fact, I’m just going to go ahead and recommend all of the optimization manuals from Agner Fog. Also, hardware vendors provide documentation for their particular hardware.
Learning to Use Tools
Having a good mental model of C++ and hardware performance is very useful when optimizing code. But, learning to use the right tools is at least as useful. Arguably the best advice for optimization is “Measure first!”. It is extremely difficult to understand the performance of even a simple block of code just by thinking about it. You’ll gain a lot of information by running the code and measuring it in various ways.
These are some of the common useful measurements:
I won’t get into recommendations for specific tools, since I am arguably getting out of scope of your original question. And, tooling is a big topic by itself: tools vary for different hardware platforms, software platforms, and by cost (some are free, some are expensive).
But, you certainly need to be aware that to optimize C++ code, you need to know and use appropriate tools.