Question

Firstly: sorry about my pathetic english language skill.
Secondly: i have learnt python for few weeks, so please be patient.:)

My project:
I got a directory with multiple files (xml).

they looks like:    
            thingy1_442354565666.xml
            thingy2_502354565666.xml
            thingy3_692354565666.xml
            etc.

The numbers in filename represent the files last modification date. The bigger number means recent files. (This will be important later.)

Each xml contains two element.

for example:
        <period>012012</period>     # this represent months
        <charges>1098</charges>     # EUR

The problem:
I need an output txt with from each month.

like this:  
        jan: 1098
        feb: 499
        etc.

Unfortunately I have more than 12 xml in that directory, so i got 2-3 files (I always need the last modified file) in each months.

I tried to make a dictionary with filenames and period element, but i totaly lost.
Btw i can parse xml with elementtree etc., but i dont know how to choose the last modified file from each month.

Please help me out, and ask if i was not understandable.
Thanks!

Answer 1

If larger numbers mean older files, sort the file list and iterate the files from smallest to largest (i.e. newest files first).

Parse the xml and keep track of the (year, month) pairs you have seen. If you have already seen a pair, then you have already processed the newest file for that month and you can ignore subsequent files.

Something like:

import glob
from datetime import datetime
from lxml import etree

def file_timestamp(filepath):
    """Return the timestamp from a file name"""
    filename = os.path.split(filepath)[1]
    filename = os.path.splitext(filename)[0]
    if '_' in filename:
        return filename.split('_')[1]
    return None

class Datafiles(object):
    def __init__(self, dir_path):
        self.date_cache = {}
        self.dir_path = dir_path

    def __iter__(self):
        return self.files()

    def next(self):
        return self.files().next()

    def files(self):
        filepaths = glob.glob(self.dir_path + '*_*.xml')
        filepaths = sorted(filepaths, key=file_timestamp)

        for filepath in filepaths:
            ## Parse data (xml)
            data = self.parse_file(filepath)
            ## If year/month is seen before, skip
            date = datetime.strptime(data['period'].text, '%m%Y')
            month = (date.year, date.month)
            if month in self.date_cache:
                continue

            ## Else store date and yield
            self.date_cache[month] = filepath
            yield data

    def parse_file(self, filepath):
        # YOUR XML PARSING HERE
        return etree.parse(filepath)

Usage:

>>> files = Datafiles(data_directory)
>>> for xml_data in files:
...     ## do something with the data