Question

I would like to do the following:

1. read multiple ascii files (*.txt) into memory. I need to skip the first 4 lines of each file. The integer values in these files are written unstructured and continuous line after line. The number of values per line can vary and is not important.
2. Once I have read all data in memory I need to write it to one single file and one value per line, i.e. one column. Before writing the values, I need to write 3 lines of header.

I need to have this as script which I can call from the terminal (Bash). I appreciate your help!

Answer 1

Your question is very vague, but I came up with this:

These are my assumptions:

1. The “values” in each line of the text file are separated by whitespace
2. The output file will contain values in the order in which the files they are contained in were passed in. So, if the function’s params were f1, f2, f3 and outfilepath, then the values in f1 appear first in outfilepath and the values in f2 appear second in outfilepath, etc

Here is a function that does what you want under these assumptions:

def readwrite(infilepaths, outfilepath):
    with open(outfilepath, 'w') as outfile:
        outfile.write(threeLinesOfHeader + '\n')
        for infilepath in infilepaths:
            with open(infilepath) as infile:
                skipLines = 4
                for _ in range(skipLines):
                    infile.readline()
                values = itertools.chain.from_iterable(line.strip().split() for line in infile)
                outfile.write('\n'.join(values) + '\n')
    print 'done'

Hope this helps