Question

Well, I understand what kind of tool I could use for optimization, profiling etc… so now I know all my bottle necks. But I can’t optimize them infinitely. What I mean ? For example, I have spot A as my main bottle neck, i applied optimization and after that CPU time, memory time etc become smaller, but related to whole project it is still bottle neck. I mean that everytime in any project there is bottle neck, so where is final point when I should stop my optimization ?

Answer 1

Suppose you find a “bottleneck” that, when you fix it, saves 30% of the time.
e.g. the time goes from 10s down to 7s.

Then suppose you find another one that, when you fix it, saves 30% of the 7s, or 2.1s.
That brings the time down to 4.9s.

Notice a couple things.

• The second problem was only 21% of the original 10s, so originally it was a less-big problem, but after fixing the first problem the second one got bigger, as a percent.

• Look at it as a speedup ratio. Fixing the first problem gave you a speedup ratio of 10/7 or 1.43x. Fixing the second one gave you a ratio of 7/4.9 or 1.43x. The total speedup ratio was 10/7 times 7/4.9, which equals 10/4.9 or 2.04x. You have more than doubled the program’s speed.

• Those two problems took 30% and 21% of the original 10s. If there were two more, taking 14.7% and 10.3%, they would be looking pretty small in a profile. But if you fix all four of them, you quadruple the program’s speed! If you miss any one of them, you don’t.

That’s how it goes. Until you truly hit rock-bottom, you can always make the program faster, and problems which might at first seem insignificant later become worthwhile.
The final speedup ratio you get can surprise you.

Not all profilers find all problems.
If one of the problems is one that your profiler will not find,
you will get to a point where you think you have optimized as far as possible, but you have not.

If you want to progress beyond that point, you need a profiling technique that will find every problem.
Here’s how I do it.