Question

I have several possible files which could hold my data; they can be compressed in different ways, so to open them I need to use file(), gzip.GzipFile() and other which also return a file object (supporting the with interface).

I want to try each of them until one succeeds in opening, so I could do something like

try:
  with gzip.GzipFile(fn + '.gz') as f:
    result = process(f)
except (IOError, MaybeSomeGzipExceptions):
  try:
    with xCompressLib.xCompressFile(fn + '.x') as f:
      result = process(f)
  except (IOError, MaybeSomeXCompressExceptions):
    try:
      with file(fn) as f:
        result = process(f)
    except IOError:
      result = "some default value"

which obviously isn’t feasible in case I have dozens of possible compression variants. (The nesting will get deeper and deeper, the code always looking very much alike.)

Is there a nicer way to spell this out?

EDIT: If possible I’d like to have the process(f) out of the try/except as well to avoid accidental catching of exceptions raised in the process(f).

Answer 1

I’d write a custom context manager:

from contextlib import contextmanager

filetypes = [('.gz', gzip.GzipFile, (IOError, MaybeSomeGzipExceptions)), 
             ('.x', xCompressLib.xCompressFile, (IOError, MaybeSomeXCompressExceptions))]

@contextmanager
def open_compressed(fn):
    f = None
    try:
        for ext, cls, exs in filetypes:
            try:
                f = cls(fn + ext)
            except exs:
                pass
            else:
                break
        yield f
    finally:
        if f is not None:
            f.close()

with open_compressed(fn) as f:
    result = "some default value" if f is None else process(f)

Or possibly just a function that returns a context manager:

filetypes = [('.gz', gzip.GzipFile, (IOError, MaybeSomeGzipExceptions)), 
             ('.x', xCompressLib.xCompressFile, (IOError, MaybeSomeXCompressExceptions))]

class UnknownCompressionFormat(Exception):
    pass

def open_compressed(fn):
    for ext, cls, exs in filetypes:
        try:
            return cls(fn + ext)
        except exs:
            pass
    raise UnknownCompressionFormat

try:
    with open_compressed(fn) as f:
        result = process(f)
except UnknownCompressionFormat:
    result = "some default value"