Question

There’s a text file that I’m reading line by line. It looks something like this:

3

3

67

46

67

3

46

Each time the program encounters a new number, it writes it to a text file. The way I’m thinking of doing this is writing the first number to the file, then looking at the second number and checking if it’s already in the output file. If it isn’t, it writes THAT number to the file. If it is, it skips that line to avoid repetitions and goes on to the next line. How do I do this?

Answer 1

Instead of checking output file for the number if it was already written it is better to keep this information in a variable (a set or list). It will save you on disk reads.

To search a file for numbers you need to loop through each line of that file, you can do that with for line in open('input'): loop, where input is the name of your file. On each iteration line would contain one line of input file ended with end of line character ‘\n’.

In each iteration you should try to convert the value on that line to a number, int() function may be used. You may want to protect yourself against empty lines or non-number values with try statement.

In each iteration having the number you should check if the value you found wasn’t already written to the output file by checking a set of already written numbers. If value is not in the set yet, add it and write to the output file.

#!/usr/bin/env python                                                           
numbers = set() # create a set for storing numbers that were already written       
out = open('output', 'w') # open 'output' file for writing                      
for line in open('input'): # loop through each line of 'input' file             
    try:                                                                        
        i = int(line) # try to convert line to integer                          
    except ValueError:  # if conversion to integer fails display a warning         
        print "Warning: cannot convert to number string '%s'" % line.strip()       
        continue # skip to next line on error                                   
    if i not in numbers: # check if the number wasn't already added to the set  
        out.write('%d\n' % i) # write the number to the 'output' file followed by EOL
        numbers.add(i) # add number to the set to mark it as already added

This example assumes that your input file contains single numbers on each line. In case of empty on incorrect line a warning will be displayed to stdout.

You could also use list in the above example, but it may be less efficient.
Instead of numbers = set() use numbers = [] and instead of numbers.add(i): numbers.append(i). The if condition stays the same.