Question

I use scrapy to take information from a website that according to w3 validator is utf-8..

My python project has

# -*- coding: utf-8 -*-

I receive some names like López J and when I print it, it shows fine…

But when I want to store it into the mysql I get some error about ascii not being able to encode blah blah blah…

If I use .encode ('ascii', 'ignore') i get: Lpez J
If I use .encode ('ascii', 'replace') i get: López J
if I use .encode ('utf-8') i get: López J

What should I do?
I’m in a big trouble here :'(

Answer 1

When you connect to the database use charset='utf-8', use_unicode=True with other keywords to connect() method. This should make the dababase accept and return unicode values, so you don’t have to (and shouldn’t) encode them manually.

Example:

>>> import MySQLdb
>>> conn = MySQLdb.connect(... , use_unicode=True, charset='utf8')
>>> cur = conn.cursor()
>>> cur.execute('CREATE TABLE testing(x VARCHAR(20))')
0L
>>> cur.execute('INSERT INTO testing values(%s)', ('López J',))
1L
>>> cur.execute('SELECT * FROM testing')
1L
>>> print cur.fetchall()[0][0]
López J