I have 13 numbers drawing from a set with 13 types of data, each type has 4 item so total 52 items. We can number the item as 1,2,3,4,5,6,7,8,9,10,11,12,13, so there will be 4 “1”, 4″2″, … 4″13″ in the set. The 13 numbers drawing from the set are random. The whole process repeated million times or even more, so I need a efficient way to store the 13 numbers. I was thinking to use some sort of coding method to compress the 13 integers into bits. For example, I count how many “1”, “2” … first, coding the count for each item with 2 bits and use 1 more bit to denote if the item was drawn or not. So for each item, we need 3 bits, total 13 items cost 39 bits. It definite need 8 bytes to do so. But it is still too much since I am talking about couple millions or billion times of calculation and each set have to be stored to the file later. So if I use 8 bytes, if will still asking about 80GB for my data. However, if I can reduce that by half, I will save 40GB. Any idea how to compress this structure more efficiently? I also think of to use 5 bytes instead but than I need to take care of the different type of number (one int + one char), is there any library in c++ can easily do the coding/compressing for me?
Thanks.
try googling LV77 and LVZ compression