Question

I’m trying to open a number of files using glob and feed them through a series of functions. Some of my files are gziped some are bz2 and some are plain text. I used fileinput typically but can’t figure out the syntax to have it take in compressed files. Based on this Python Fileinput Doc it should be something like:

openhook=fileinput.hook_compressed

My code looks like:

import fileinput
import glob

filestobeanalyzed = glob.glob('./files/*')

for fileName in filestobeanalyzed:
    inputfilename = fileName
    for line in fileinput.input([inputfilename, openhook=fileinput.hook_compressed]):
        #do stuff

I get an invalid syntax on the fileinput line at the = sign.

Any suggestions?

Answer 1

You want

for line in fileinput.input(inputfilename, openhook=fileinput.hook_compressed):
    #do stuff

(I removed the square brackets). You were trying to do an assignment in a list constructor. e.g.

my_list=["foo",bar="baz"]  #this doesn't work (SyntaxError)

You probably got the idea from the python documentation which uses [ and ] to indicate optional arguments to functions.

This is just an aside — often there is more information in the traceback which can help pin down the problem than just the type of error and the line number. (read: When you have a traceback, it’s generally appreciated if you paste the whole thing so we can see it)