Question

I’m trying to use Intel intrinsics to beat the compiler optimized code. Sometimes I can do it, other times I can’t.

I guess the question is, why can I sometimes beat the compiler, but other times not? I got a time of 0.006 seconds for operator+= below using Intel intrinsics, (vs 0.009 when using bare C++), but a time of 0.07 s for operator+ using intrinsics, while bare C++ was only 0.03 s.

#include <windows.h>
#include <stdio.h>
#include <intrin.h>

class Timer
{
  LARGE_INTEGER startTime ;
  double fFreq ;

public:
  Timer() {
    LARGE_INTEGER freq ;
    QueryPerformanceFrequency( &freq ) ;
    fFreq = (double)freq.QuadPart ;
    reset();
  }

  void reset() {   QueryPerformanceCounter( &startTime ) ;  }

  double getTime() {
    LARGE_INTEGER endTime ;
    QueryPerformanceCounter( &endTime ) ;
    return ( endTime.QuadPart - startTime.QuadPart ) / fFreq ; // as double
  }
} ;


inline float randFloat(){
  return (float)rand()/RAND_MAX ;
}



// Use my optimized code,
#define OPTIMIZED_PLUS_EQUALS
#define OPTIMIZED_PLUS

union Vector
{
  struct { float x,y,z,w ; } ;
  __m128 reg ;

  Vector():x(0.f),y(0.f),z(0.f),w(0.f) {}
  Vector( float ix, float iy, float iz, float iw ):x(ix),y(iy),z(iz),w(iw) {}
  //Vector( __m128 val ):x(val.m128_f32[0]),y(val.m128_f32[1]),z(val.m128_f32[2]),w(val.m128_f32[3]) {}
  Vector( __m128 val ):reg( val ) {} // 2x speed, above

  inline Vector& operator+=( const Vector& o ) {
    #ifdef OPTIMIZED_PLUS_EQUALS
    // YES! I beat it!  Using this intrinsic is faster than just C++.
    reg = _mm_add_ps( reg, o.reg ) ;
    #else
    x+=o.x, y+=o.y, z+=o.z, w+=o.w ;
    #endif
    return *this ;
  }

  inline Vector operator+( const Vector& o )
  {
    #ifdef OPTIMIZED_PLUS
    // This is slower
    return Vector( _mm_add_ps( reg, o.reg ) ) ;
    #else
    return Vector( x+o.x, y+o.y, z+o.z, w+o.w ) ;
    #endif
  }

  static Vector random(){
    return Vector( randFloat(), randFloat(), randFloat(), randFloat() ) ;
  }

  void print() {

    printf( "%.2f %.2f %.2f\n", x,y,z,w ) ;
  }
} ;

int runs = 8000000 ;
Vector sum ;

// OPTIMIZED_PLUS_EQUALS (intrinsics) runs FASTER 0.006 intrinsics, vs 0.009 (std C++)
void test1(){
  for( int i = 0 ; i < runs ; i++ )
    sum += Vector(1.f,0.25f,0.5f,0.5f) ;//Vector::random() ;
}

// OPTIMIZED* runs SLOWER (0.03 for reg.C++, vs 0.07 for intrinsics)
void test2(){
  float j = 27.f ;
  for( int i = 0 ; i < runs ; i++ )
  {
    sum += Vector( j*i, i, i/j, i ) + Vector( i, 2*i*j, 3*i*j*j, 4*i ) ;
  }
}

int main()
{
  Timer timer ;

  //test1() ;
  test2() ;

  printf( "Time: %f\n", timer.getTime() ) ;
  sum.print() ;

}

Edit

Why am I doing this? The VS 2012 profiler is telling me my vector arithmetic operations could use some tuning.

Answer 1

As noted by Mysticial, the union hack is the most likely culprit in test2. It forces the data to go through L1 cache, which, while fast, has some latency that is much more than your gain of 2 cycles that the vector code offers (see below).

But also consider that the CPU can run multiple instructions out of order and in parallel (superscalar CPU). For example, Sandy Bridge has 6 execution units, p0–p5, floating point multiplication/division runs on p0, floating point addition and integer multiplication runs on p1. Also, division takes 3-4 times more cycles then multiplication/addition, and is not pipelined (i.e. the execution unit cannot start another instruction while division is being performed). So in test2, while the vector code is waiting for the expensive division and some multiplications to finish on unit p0, the scalar code can be performing the extra 2 add instructions on p1, which most likely obliterates any advantage of vector instructions.

test1 is different, the constant vector can be stored in xmm register and in that case the loop contains only the add instruction. But the code is not 3x faster as might be expected. The reason is pipelined instructions: each add instruction has latency 3 cycles, but the CPU can start a new one every cycle when they are independent of each other. This is the case of the per-component vector addition. Therefore the vector code executes one add instruction per loop iteration with 3 cycle latency, and the scalar code executes 3 add instructions, taking only 5 cycles (1 started/cycle, and the 3rd has latency 3: 2 + 3 = 5).

A very good resource on CPU architectures and optimization is http://www.agner.org/optimize/