I have written a C++ code to calculate the reflection of solar radiation (based on ray tracing principals).
I have included a number of acceleration techniques.
In my write up I have to justify these algorithms. I was going to do it purely from a time base but the comment by @weberc2 HERE makes be believe that is not the best solution.
I have looked at code analysis software like Very sleepy and AMD Code Analyst which helped to identify bottle necks etc.
As the supervisor will probably have a very poor programming knowledge a time based analysis just seems the most logical…
e.g
“Running the same scenario with the Grid active increased calculation accuracy by 20% with only a 2sec penalty on time…”
It is a single thread program.
Is it really that dangerous to use time?
Any suggestions.
Thank you all
I disagree. Comparing times is perfectly fine – but with a limitation.
A single run says nothing. This is why we have statistical tools and tests to show
A is distinct then B.Run a series of tests, on several test cases and various conditions. Store the data (run times) in two different lists, and then run a statistical test to show one is better then the other.
The “answer” from the statistic test is a P-Value. The P-Value says “What is the probability that you are wrong”. For example, if you had a set of tests, and you run a statisitcal test and find
P_Value = 0.01. This means that with probability 99% – the two samples are distinct, and you can conclude the one with the lower average is better.The de-facto standard for statistical tests (at least in my field) is Wilcoxon Paired Signed Test.
P.S. The statistical test will “prove” the assumption for the conditions it was tested on, for example – if you run it on AMD CPU, it says nothing on what will happen on Intel CPU (maybe the instruction set makes the “worse” significantly better in it).
However, note that nevertheless, it is very acceptable and widely used in articles in fields such as AI and Information Retrieval.