Question

Given a parallel program and a well-defined hardware, how to determine its compute-to-communication ratio (CCR). This is to determine a programs’ granularity and subsequent analysis. Literature describes this term and uses it to relate to a programs’ granularity but I haven;t found anything to estimate the CCR if it is high or low.

Answer 1

To compare compute and communication you have to measure both in a common unit. Generally I would use a measure of time. You might then find that a program has a ratio of 80:20 (compute:communicate)

While one might assert that a ratio of 50:50 is a high value for communication, these figures really only take meaning in relation to comparable figures for other parallel programs. So, if program A and program B solve the same problem but A has a ratio of 80:20 and B of 70:30, then you can make the statement that A does more computation than B in each tick of the clock. If they’re not solving the same problem you may find yourself comparing apples and oranges.

And while computation is generally preferred to communication, again these factors are relative. Given the programs A and B, sure B does (relatively) more communication, but it is possible also that B takes less time overall than A — perhaps B spends time sending larger messages between processes while A does more duplicate computation to avoid sending larger messages. Choose which program you prefer on whatever grounds you like.

If you have enough experience of these matters you can usually make some educated guesses about the ratio from examining the program structure, but in the end you are going to have to measure it to draw any sensible conclusions.