I do some numerical computing, and I have often had problems with floating points computations when using GCC. For my current purpose, I don’t care too much about the real precision of the results, but I want this firm property:
no matter WHERE the SAME code is in my program, when it is run on the SAME inputs, I want it to give the SAME outputs.
How can I force GCC to do this? And specifically, what is the behavior of –fast-math, and the different -O optimizations?
I’ve heard that GCC might try to be clever, and sometimes load floats in registers, and sometime read them directly from memory, and that this might change the precision of the floats, resulting in a different output. How can I avoid this?
Again, I want :
- my computations to be fast
- my computations to be reliable (ie. same input -> same result)
- I don’t care that much about the precision for this particular code, so I can be fine with reduced precision if this brings reliability
could anyone tell me what is the way to go for this problem?
I think that GCC is pretty well documented so I’m not going to reveal my own ignorance by trying to answer the parts of your question about its options and their effects. I would, though, make the general statement that when numeric precision and performance are concerned, it pays big dividends to read the manual. The clever people who work on GCC put a lot of effort into their documentation, reading it is rewarding (OK, it can be a trifle dull, but heck, it’s a compiler manual not a bodice-ripper).
If it is important to you that you get identical-to-the-last-bit numeric results you’ll have to concern yourself with more than just GCC and how you can control its behaviour. You’ll need to lock down the libraries it calls, the hardware it runs on and probably a number of other factors I haven’t thought of yet. In the worst (?) case you may even want to, and I’ve seen this done, write your own implementations of f-p maths to guarantee bit-identity across platforms. This is difficult, and therefore expensive, and leaves you possibly less certain of the correctness of your own code than of the code usd by GCC.
However, you write
which prompts the question to you — why don’t you simply use 5-decimal-digit precision as your standard of (reduced) precision ? It’s what an awful lot of us in numerical computing do all the time; we ignore the finer aspects of numerical analysis since they are difficult, and costly in computation time, to circumvent. I’m thinking of things like interval arithmetic and high-precision maths. (OF course, if 5 is not right for you, choose another single-digit number.)
But the good news is that this is entirely justifiable: we’re dealing with scientific data which, by its nature, comes with errors attached (of course we generally don’t know what the errors are but that’s another matter) so it’s OK to disregard the last few digits in the decimal representation of, say, a 64-bit f-p number. Go right ahead and ignore a few more of them. Even better, it doesn’t matter how many bits your f-p numbers have, you will always lose some precision doing numerical calculations on computers; adding more bits just pushes the errors back, both towards the least-significant-bits and towards the end of long-running computations.
The case you have to watch out for is where you have such a poor algorithm, or a poor implementation of an algorithm, that it loses lots of precision quickly. This usually shows up with any reasonable size of f-p number. Your test suite should have exposed this if it is a real problem for you.
To conclude: you have to deal with loss of precision in some way and it’s not necessarily wrong to brush the finer details under the carpet.