Question

Recently, I started writing a program to compare DNA sequence. As the alphabet only consists of four letters (ATCG), compressing each character to 2 bits seemed like it would offer faster comparisons (are two characters the same or different). However, when I ran a test char comparisons were much faster than bit comparisons (by ~30%). Compression was carried out in both programs as a control. What am I missing here? Is there a more efficient way to compare bits?
p.s. I also tried vector, but it was a bit slower than bitset.

// File: bittest.cc
// Test use of bitset container

#include <ctime>
#include <iostream>
#include <bitset>
#include <vector>
#include <string>
using namespace std;

void compress(string&, bitset<74>&);
void compare(bitset<74>&, bitset<74>&);

int main()
{
   // Start timer
   std::clock_t start;
   double difference;
   start = std::clock();

   for(int i=0; i<10000000; ++i){
      string frag1="ATCGACTGACTGACTGACTGACTGACTGACTGACTGA";
      string frag2="AACGAACGAACGAACGAACGAACGAACGAACGAACGA";
      int a=37;
      bitset<74> bits1;
      bitset<74> bits2;
      compress(frag1, bits1);
      compress(frag2, bits2);
      compare(bits1, bits2);
   }

   difference = ( std::clock() - start ) / (double)CLOCKS_PER_SEC;
   int minutes = difference/60;
   int seconds = difference - minutes * 60;

   if (seconds < 10){
      cout << "\nRunning time: " << minutes << ":0" << seconds << endl << endl;
   }else{
      cout << "\nRunning time: " << minutes << ":" << seconds << endl << endl;
   }

   return 0;
}

void compress(string& in, bitset<74>& out){
   char c;
   int b=0;
   for(int i=0; i<in.length(); ++i){
      c=in[i];
      b=2*i;
      switch(c){
        case 'A':
           break;
        case 'C':
           out.set(b+1);
           break;
        case 'G':
           out.set(b);
           break;
        case 'T':
           out.set(b);
           out.set(b+1);
           break;
        default:
           cout << "Invalid character in fragment.\n";
      }
   }
}

void compare(bitset<74>& a, bitset<74>& b){
   for(int i=0; i<74; ++i){
      if(a[i] != b[i]){
      }
   }
}

And the string harness…

// File: bittest.cc

#include <ctime>
#include <iostream>
#include <bitset>
#include <vector>
#include <string>
using namespace std;

void compress(string&, bitset<74>&);
void compare(string&, string&);

int main()
{
   // Start timer
   std::clock_t start;
   double difference;
   start = std::clock();

   for(int i=0; i<10000000; ++i){
      string frag1="ATCGACTGACTGACTGACTGACTGACTGACTGACTGA";
      string frag2="AACGAACGAACGAACGAACGAACGAACGAACGAACGA";
      int a=37;
      bitset<74> bits1;
      bitset<74> bits2;
      compress(frag1, bits1);
      compress(frag2, bits2);
      compare(frag1, frag2);
   }

   difference = ( std::clock() - start ) / (double)CLOCKS_PER_SEC;
   int minutes = difference/60;
   int seconds = difference - minutes * 60;

   if (seconds < 10){
      cout << "\nRunning time: " << minutes << ":0" << seconds << endl << endl;
   }else{
      cout << "\nRunning time: " << minutes << ":" << seconds << endl << endl;
   }

   return 0;
}

void compress(string& in, bitset<74>& out){
   char c;
   int b=0;
   for(int i=0; i<in.length(); ++i){
      c=in[i];
      b=2*i;
      switch(c){
        case 'A':
           break;
        case 'C':
           out.set(b+1);
           break;
        case 'G':
           out.set(b);
           break;
        case 'T':
           out.set(b);
           out.set(b+1);
           break;
        default:
           cout << "Invalid character in frag.\n";
      }
   }
}

void compare(string& a, string& b){
   for(int i=0; i<37; ++i){
      if(a[i] != b[i]){
      }
   }
}

Answer 1

Consider the two comparison routines:

void compare(bitset<74>& a, bitset<74>& b){
   for(int i=0; i<74; ++i){
      if(a[i] != b[i]){
      }
   }
}

and

void compare(string& a, string& b){
   for(int i=0; i<37; ++i){
      if(a[i] != b[i]){
      }
   }
}

Right off the bat you can see that one is executing the loop 74 times and the other is executing the loop 37 times. So already the bitset approach is starting from a position of weakness.

Now consider the types of data being accessed; accessing individual bytes is reasonably quick; accessing individual bits from any data structure might store a single bit in an entire byte or maybe even some larger word size. If it stores bits in individual bits, then some bitmasking operations must be introduced, and those all take processing power too. If the bits are stored in bytes, then you are in fact comparing only half of each character on each bit. If the bits are stored in words or larger, you are increasing the size of the CPU’s data cache — potentially taking something that might fit entirely in one cache line to several cache lines. That’s a potential for gigantic speed penalties, though on inputs this small, it’s probably not too horrible yet.

If you replace your bitset with a char[] that is large enough to hold all your data, manually set the bits yourself in the compression routines, and then compare the char[] array a byte at a time or larger, you can probably drastically improve the speed of the comparison routines. Will the speed up be sufficient to overcome the cost of the compression routines? That’s tough to say and depends in part upon how many comparisons you can make with each compressed form.

If you can perform your comparison using int or larger datatypes, you can probably go even significantly faster, as modern CPUs are usually faster at accessing 4-bytes or 8-bytes at a time than 1-byte at a time. Most strcmp(3) or memcmp(3) routines are optimized to perform huge, aligned reads. If you use memcmp(3) to do your comparison, you’ll have the best chance of going at top speed — and that goes for both the compressed and uncompressed versions.