Question

I have written a python script to parse an HTML page, get some strings and then write to a mysql table. I am using the MySQLDb module for the database connection.
The strings retrieved are encoded in ISO-8859-7 (Greek), which is the default encoding in the MySQL table as well. The code which produces the exception is the following :

  def db_write(list) :
    import MySQLdb as sql
    try :
        con = sql.connect(//database info here//)
    except :
        print "could not connect to database"
        exit()
    cur = con.cursor()
    for i in my_range (8,len(list)-2,2) :
        query = 'INSERT INTO as_doy VALUES (%s,"%s")' % (list[i],list[i+1])
        print query        
        try :
            cur.execute(query)
            con.commit()
        except :
            print "failed"
            con.rollback()
    con.close()

The exception i get is ERROR 1366 (HY000): Incorrect string value: '\xEF\xBF\xBD\xEF\xBF\xBD...'

I have tried encoding the strings in utf-8, decoding and re-encoding in iso-8859-7, but nothing has worked for me yet.

Answer 1

Make sure your connection collation is correct by passing the charset argument to .connect() or execute SET NAMES utf8 (or equivalent) before you run any queries.

For more tips, see this link.