Question

import csv

with open('thefile.csv', 'rb') as f:
  data = list(csv.reader(f))
  import collections
  counter = collections.defaultdict(int)

  for row in data:
        counter[row[10]] += 1


with open('/pythonwork/thefile_subset11.csv', 'w') as outfile:
    writer = csv.writer(outfile)
    for row in data:
        if counter[row[10]] >= 504:
           writer.writerow(row)

This code reads thefile.csv, makes changes, and writes results to thefile_subset1.

However, when I open the resulting csv in Microsoft Excel, there is an extra blank line after each record!

Is there a way to make it not put an extra blank line?

Answer 1

The csv.writer module directly controls line endings and writes \r\n into the file directly. In Python 3 the file must be opened in untranslated text mode with the parameters 'w', newline='' (empty string) or it will write \r\r\n on Windows, where the default text mode will translate each \n into \r\n.

#!python3
with open('/pythonwork/thefile_subset11.csv', 'w', newline='') as outfile:
    writer = csv.writer(outfile)

If using the Path module:

from pathlib import Path
import csv

with Path('/pythonwork/thefile_subset11.csv').open('w', newline='') as outfile:
    writer = csv.writer(outfile)

If using the StringIO module to build an in-memory result, the result string will contain the translated line terminator:

from io import StringIO
import csv

s = StringIO()
writer = csv.writer(s)
writer.writerow([1,2,3])
print(repr(s.getvalue()))  # '1,2,3\r\n'   (Windows result)

If writing that string to a file later, remember to use newline='':

# built-in open()
with open('/pythonwork/thefile_subset11.csv', 'w', newline='') as f:
    f.write(s.getvalue())

# Path's open()
with Path('/pythonwork/thefile_subset11.csv').open('w', newline='') as f:
    f.write(s.getvalue())

# Path's write_text() added the newline parameter to Python 3.10.
Path('/pythonwork/thefile_subset11.csv').write_text(s.getvalue(), newline='')

In Python 2, use binary mode to open outfile with mode 'wb' instead of 'w' to prevent Windows newline translation. Python 2 also has problems with Unicode and requires other workarounds to write non-ASCII text. See the Python 2 link below and the UnicodeReader and UnicodeWriter examples at the end of the page if you have to deal with writing Unicode strings to CSVs on Python 2, or look into the 3rd party unicodecsv module:

#!python2
with open('/pythonwork/thefile_subset11.csv', 'wb') as outfile:
    writer = csv.writer(outfile)

Documentation Links

• https://docs.python.org/3/library/csv.html#csv.writer
• https://docs.python.org/2/library/csv.html#csv.writer