Question

I tried coming up with a compression algorithm. I do little bit about compression theories and so am aware that this scheme that I have come up with could very well never achieve compression at all.

Currently it works only for a string with no consecutive repeating letters/digits/symbols. Once properly established I hope to extrapolate it to binary data etc. But first the algorithm:

Assuming there are only 4 letters: a,b,c,d; we create a matrix/array corresponding to the letters. Whenever a letter is encountered, the corresponding index is incremented so that the index of the last letter encountered is always largest. We incremement an index by 2 if it was originally zero. If it was not originally zero then we increment it by 2+(the second largest element in the matrix). An example to clarify:

Array = [a,b,c,d]
Initial state = [0,0,0,0]
Letter = a
New state = [2,0,0,0]
Letter = b
New state = [2,4,0,0]
.
.c
.d
.
New state = [2,4,6,8]
Letter = a
New state = [12,4,6,8] 
//Explanation for the above state: 12 because Largest - Second Largest - 2 = Old value
Letter = d
New state = [12,4,6,22]
and so on...

Decompression is just this logic in reverse.

A rudimentary implementation of compression (in python):

(This function is very rudimentary so not the best kind of code…I know. I can optimize it once I get the core algorithm correct.)

def compress(text):
    matrix = [0]*95     #we are concerned with 95 printable chars for now
    for i in text:
        temp = copy.deepcopy(matrix) 
        temp.sort()
        largest = temp[-1]
        if matrix[ord(i)-32] == 0:
            matrix[ord(i)-32] = largest+2
        else:
            matrix[ord(i)-32] = largest+matrix[ord(i)-32]+2
    return matrix

The returned matrix is then used for decompression. Now comes the tricky part:

I can’t really call this compression at all because each number in the matrix generated from the function are of the order of 10**200 for a string of length 50000. So storing the matrix actually takes more space than storing the original string. I know…totally useless. But I had hoped prior to doing all this that I can use the mathematical properties of a matrix to effectively represent it in some kind of mathematical shorthand. I have tried many possibilities and failed. Some things that I tried:

1. Rank of the matrix. Failed because not unique.

2. Denote using the mod function. Failed because either the quotient or the remainder

3. Store each integer as a generator using pickle.

4. Store the matrix as a bitmap file but then the integers are too large to be able to store as color codes.

Let me iterate again that the algorithm could be optimized. e.g. instead of adding 2 we could add 1 and proceed. But don’t really result in any compression. Same for the code. Minor optimizations later…first I want to improve the main algorithm.

Furthermore, it is very likely that this product of a mediocre and idle mind like myself could never be able to achieve compression after all. In which case, I would then like your help and ideas on what this could probably be useful in.

TL;DR: Check coded parts which depict a compression algorithm. The compressed result is longer than the original string. Can this be fixed? If yes, how?

PS: I have the entire code on my PC. Will create a repo on github and upload in some time.

Answer 1

Compression is essentially a predictive process. Look for patterns in the input and use them to encode the more likely next character(s) more efficiently than the less likely. I can’t see anything in your algorithm that tries to build a predictive model.