Question

I have to remove the first 2 headers of many files and replace them with another one. As I am new to Python and programming, I have been using the following code

import glob
import os
list_of_files = glob.glob('./*.txt')
for file_name in list_of_files:
    os.system('sed "1,2d" %s | sort -k1 > %s.sort' %(file_name,file_name))
    os.system ('cat header file %s.sort > %s.header' %(file_name,file_name))

which works. However I think there should be a better way of getting this done. Also I am unnecessarily making an extra file *.sort which I don’t require.

Answer 1

Believe it or not, you can do this pretty easily in pure python:

import itertools
with open(filename) as fin:
    ilines = itertools.islice(fin, 2, None) #this throws away the first 2 lines
    lines = sorted(ilines, key=lambda x: x.split()[0])  #sort lexicographically on first column

with open('header') as header, open('%s.header'%filename) as fout:
    fout.writelines(header) #write the header
    fout.writelines(lines) #write the data

You’re done. Take a slightly longer lunch break because python helps you save time* :-).

*(Alternatively, spend some of your long lunch learning more of the cool things python has to offer!)

Happy Coding!