Question

A compressed file can be classified into below logical groups
a. The operating system which you are working on (*ix, Win) etc.
b. Different types of compression algorithm (i.e .zip,.Z,.bz2,.rar,.gzip). Atleast from a standard list of mostly used compressed files.
c. Then we have tar ball mechanism – where I suppose there are no compression. But it acts more like a concatenation.

Now, if we start addressing the above set of compressed files,
a. Option (a) would be taken care by python since it is platform independent language.
b. Option (b) and (c) seems to have a problem.

What do I need
How do I identify the file type (compression type) and then UN-compress them?

---

Like:

fileType = getFileType(fileName)  
switch(fileType):  
case .rar:  unrar....
case .zip:  unzip....

etc  

---

So the fundamental question is how do we identify the compression algorithm based on the file (assuming the extension is not provided or incorrect)? Is there any specific way to do it in python?

Answer 1

This page has a list of “magic” file signatures. Grab the ones you need and put them in a dict like below. Then we need a function that matches the dict keys with the start of the file. I’ve written a suggestion, though it can be optimized by preprocessing the magic_dict into e.g. one giant compiled regexp.

magic_dict = {
    "\x1f\x8b\x08": "gz",
    "\x42\x5a\x68": "bz2",
    "\x50\x4b\x03\x04": "zip"
    }

max_len = max(len(x) for x in magic_dict)

def file_type(filename):
    with open(filename) as f:
        file_start = f.read(max_len)
    for magic, filetype in magic_dict.items():
        if file_start.startswith(magic):
            return filetype
    return "no match"

This solution should be cross-plattform and is of course not dependent on file name extension, but it may give false positives for files with random content that just happen to start with some specific magic bytes.