Question

I’ve searched on this for a while but none seem to be quite relevant enough.

Basically, I have a csv file that is extracted from a program. I wrote a quick python program to convert the csv file to an sql file so I can import it into the database. I then import the .sql file using the windows command prompt to my remote MySQL server.

I have some pound signs (£) in some of the fields in my database. In the csv file, they are showing as £, in the resulting sql file (after conversion) they are showing as \£ because of my use of re.escape – I’m actually thinking this may be the reason (The python code that writes this to the file is:
for row in csvReader:
…other stuff…
sqlfile.write(re.escape(row[6])).

After the file is imported, I view the tables and the pound signs are now showing as ? in the tables, and when the contents is shown on the website using php to query.
The database Server connection collation is utf8_general_ci and I’m connecting to the remote server with thiscommand:
mysql –host=HOST –user=USER –password=PASS –default-character-set=utf8 DATABASENAME < filetoimport.sql

Changing the character encoding on firefox made no difference, as expected – whatever is happening seems to be when the sql dump is imported.

Being honest, I’m not really completely sure I understand how the whole character set things work, I’ve never really had to try and fix a problem like this before so I’m not really sure what else to try. Any help or even a point in the right direction to search, as my searches so far doesn’t seem to be helping me 🙁

Thank you!

Answer 1

Use mysql’s escape function. Mysqldb is something like:

conn.escape_string()