I know there are memory caches so that using values from a lot of disparate points in memory causes cache misses and hurts performance. As a program is executed is it actually loaded from memory into the cpu in small blocks similar to an L1 cache? I ask because presumably then jumping around in memory frequently would cause misses on this cache and hurt performance. So I guess it is two questions: is there such an “execution cache” and does jumping around frequently hurt performance.
P.S. Not sure what appropriate tags are for this other than performance and caching.
P.P.S. An example situation might be an otherwise tight loop containing a lot of nested ifs and elses where the end result at the bottom of each if and else nest is only a small amount of code.
Yes. This cache is called the instruction cache. Exhausting it can have severe implications for performance. This is the reason that inlining functions is not always beneficial. The inlined functions will be faster but the bloated code size might drive the hot path out of the I-cache.
The Linux kernel deverlopers mailing list has interesting discussions about this topic.