Question

I have a short method in my code to normalize a vector (actually a PCL point) which produces results of low accuracy. The code:

void normalize(pcl::PointXYZ::PointXYZ * p){
  float nf = 1/sqrt(p->x*p->x+p->y*p->y+p->z*p->z);
  //nf is a normalization factor precalculated to eliminate two FP divisions.
  p->x*=nf; p->y*=nf; p->z*=nf;
}

This function is passed the point with coordinates (-0.850650787, 1.37638187, -0.525731087). Debugging shows that nf=0.587785244 after evaluation of the second line. When I do the same calculation in Mathematica, nf=0.617708029. This is an error of more than 5%! The coordinates of p are never greater than 2 or less than -2. Is this inaccuracy typical for these operations, or is there something wrong?

Answer 1

According to my calculations, 0.587785244 is the correct result (I get 0.5877852727698576 using Perl). I suspect you’re doing the calculation incorrectly in Mathematica.