Question

I am reading a file, line-by-line and doing some text processing in order to get output in a certain format
My string processing code goes as follows:

file1=open('/myfolder/testfile.txt')
scanlines=file1.readlines()
string = ''

 for line in scanlines:
    if line.startswith('>from'):
         continue
    if line.startswith('*'):
        continue
    string.join(line.rstrip('\n')) 

The output of this code is as follows:

abc

def

ghi

Is there a way to join these physical lines into one logical line, e.g:

abcdefghi

Basically, how can I concatenate multiple strings into one large string?

If I was reading from a file with very long strings is there the risk of an overflow by concatenating multiple physical lines into one logical line?

Answer 1

Cleaning things up a bit, it would be easiest to append to array and then return the result

def joinfile(filename) :
   sarray = []
   with open(filename) as fd :
       for line in fd :
           if line.startswith('>from') or line.startswith('*'):
               continue
          sarray.append(line.rstrip('\n'))
   return ''.join(sarray)

If you wanted to get really cute you could also do the following:

fd = open(filename)
str = ''.join([line.rstrip('\n') for line in fd if not (line.startswith('>from') or line.startswith('*'))])

Yes of course you could read a file big enough to overflow memory.