Question

I am using python’s CSV module to iterate over the rows of a column.

What I need to do is:

1. Get the first row for column “title”
2. Remove any spanish characters (accents, Ñ)
3. Remove single quotes
4. Finally, replace spaces with dashes and convert everything to lowercase.

I got this to work with a simple test file,not a csv. I also managed to print each title in it’s own separate line.

But now I’m using this code to go over the CSV file (sorry for the VERY ugly code, I’m a newbie programmer):

import csv
import unicodedata
import ast

def strip_accents(s):
  return ''.join((c for c in unicodedata.normalize('NFD', s) if unicodedata.category(c) != 'Mn'))

dic_read = csv.DictReader(open("output.csv", encoding = "utf8"))

for line in dic_read:

    #print(line)     #I get each line of the csv file as a dictionary.
    #print(line["title"])  # I get only the "title" column on each line

    line = line.replace(' ', '-').lower()
    line = line.replace("´", "")
    line = strip_accents(line)
    fp=open("cleantitles.txt", "a")
    fp.write(line)
    fp.close()

I get the following error:

Traceback (most recent call last):
  File "C:/csvreader3.py", line 15, in <module> line = strip_accents(line)
 File "C:/csvreader3.py", line 7, in strip_accents
  return ''.join((c for c in unicodedata.normalize('NFD', s) if unicodedata.category(c) != 'Mn'))
  TypeError: must be str, not dict

I also get a similar error when I try to do a .replace only. I understand now that these methods only apply to strings.

How can I get this to work? I searched around for a way to convert a dict to a string object but that didn’t work.

Also, any criticism to optimize and make my code more readable are also welcome!

Answer 1

With the new information at hand, I think you might find this method to be simpler.

Use the inbuilt function ‘map’. I’ll leave the explanation of what ‘map’ does to the python documentation.

Here is what I think you should do

Create a function that takes a line/dict and processes it to the format you want

def strip_unwanted(line):
    title = str(line['title']).replace(' ', '-').replace("´", "")
    title = ''.join((c for c in unicodedata.normalize('NFD', s) if unicodedata.category(c) != 'Mn'))
    line['title'] = title
    return line

with open("output.csv", encoding = "utf8") as input:
    dic_entries = csv.DictReader(input)
    # use the 'map' function
    new_entries = map(strip_unwanted, dic_entries)

    with open('some.csv', 'wb') as output:
        writer = csv.DictWriter(output)
        writer.writerows(new_entries)