Question

I have a directory of files that I am trying to parse using Python. I wouldn’t have a problem if they were all the same extension, but for whatever reason they are created with sequential numeric extensions after their original extension. For example: foo.log foo.log.1 foo.log.2 bar.log bar.log.1 bar.log.2 etc. On top of that, foo.log is in XML format, while bar.log is not. What’s the best route to take in order to read and parse only the foo.log.* and foo.log files? The bar.log files do not need to be read. Below is my code:

import os
from lxml import etree
path = 'C:/foo/bar//'
listing = os.listdir(path)
for files in listing:
    if files.endswith('.log'):
        print files
        data = open(os.path.join(path, files), 'rb').read()
        tree = etree.fromstring(data)
        search = tree.findall('.//QueueEntry')

This doesn’t work as it doesn’t read any .log.* files and the parser chokes on the files that are read, but are not in xml format. Thanks!

Answer 1

Maybe the glob module can help you:

import glob

listing = glob.glob('C:/foo/bar/foo.log*')
for filename in listing:
    # do stuff