Question

The following code realizes the same computation by either using an Eigen vector as a container only or a simple C-array. It produces a closed but not bit-to-bit equivalent result.

The final mathematical operation is x * alpha + y * beta.

#include <Eigen/Eigen>

int main()
{
  Eigen::VectorXd x(2);
  double* y = new double[2];
  long long int a = 4603016991731078785;
  double ga = *(double*)(&a);
  long long int b = -4617595986472363966;
  double gb = *(double*)(&b);
  long long int x0 = 451;
  long long int x1 = -9223372036854775100;
  x[0] = *(double*)(&x0);
  y[0] = *(double*)(&x0);
  x[1] = *(double*)(&x1);
  y[1] = *(double*)(&x1);
  double r = ga*x[0] + gb*x[1];
  double s = ga*y[0] + gb*y[1];
}

Why is it so?

Results differ when using MSVC and gcc (64-bits OS).

Answer 1

This is probably because one computation is done completely within the FPU (floating-point unit) with 80 bits of precision, while the other computation uses partially 64 bits of precision (the size of a double). This can also be demonstrated without using Eigen. Look at the following program:

int main()
{
  // Load ga, gb, y[0], y[1] as in original program
  double* y = new double[2];
  long long int a = 4603016991731078785;
  double ga = *(double*)(&a);
  long long int b = -4617595986472363966;
  double gb = *(double*)(&b);
  long long int x0 = 451;
  long long int x1 = -9223372036854775100;
  y[0] = *(double*)(&x0);
  y[1] = *(double*)(&x1);

  // Compute s as in original program
  double s = ga*y[0] + gb*y[1];

  // Same computation, but in steps
  double r1 = ga*y[0];
  double r2 = gb*y[1];
  double r = r1+r2;
}

If you compile this without optimization, you will see that r and s have different values (at least, I saw that on my machine). Looking at the assembly code, in the first computation, the values of ga, y[0], gb and y[1] are loaded in the FPU, then the calculation ga * y[0] + gb * y[1] is done, and then the result is stored in memory. The FPU does all computations with 80 bits, but when the result is stored in memory the number is rounded so that it fits within the 64 bits of a double variable.

The second computation proceeds differently. First, ga and y[0] are loaded in the FPU, multiplied, and then rounded to a 64-bits number and stored in memory. Then, gb and y[1] are loaded in the FPU, multiplied, and then rounded to a 64-bits number and stored in memory. Finally, r1 and r2 are loaded in the FPU, added, rounded to a 64-bits number and stored in memory. This time, the computer rounds intermediate results, and this leads to the difference.

For this computation, rounding has a fairly large effect because you are working with denormal numbers.

Now, here comes the bit where I am not so certain (and if this was your question, I apologize): what does this have to do with the original program, where x is an Eigen container? Here the computation goes as follows: a function from Eigen is called to get x[0], then ga and the result from that function is loaded into the FPU, multiplied, and stored in a temporary memory location (64 bits, so this is rounded). Then gb and x[1] are loaded into the FPU, multiplied, added to the intermediate result stored in a temporary memory location, and finally stored in x. So in the computation of r in the original program, the result of ga*x[0] is rounded to 64 bits. Perhaps the reason for this is that the floating point stack is not preserved across function calls.