Question

I am trying to enter values into a mysql db using python. To this end, I have something like:

import MySQLdb
variables=[]*10 #list of variables.
n=len(variables)

#create python-mysql cursor connection
..
#load values into mysql:
for i in range(0,n):
    cursor.executemany(sql, variables[i])

This shoots out some errors in rows (approx 1% of the rows):

Warning: Incorrect string value: '\xD0\xA2\xD0\xB5\xD0\xBA...' for column 'Link' at row 395
cursor.executemany(sql, variables[i])
mergeDB.py:85: Warning: Incorrect string value: '\xD0\x91\xD0\xB8\xD1\x82...' for column 'Link' at row 431
cursor.executemany(sql, variables[i])
mergeDB.py:85: Warning: Incorrect string value: '\xD0\x91\xD0\xB8\xD1\x82...' for column 'Link' at row 568
cursor.executemany(sql, variables[i])
mergeDB.py:85: Warning: Incorrect string value: '\xD0\xA7\xD0\xB5\xD0\xBB...' for column 'Link' at row 665
cursor.executemany(sql, variables[i])
mergeDB.py:85: Warning: Incorrect string value: '\xD0\x96\xD0\xB5\xD0\xBB...' for column 'Link' at row 827
cursor.executemany(sql, variables[i])
mergeDB.py:85: Warning: Incorrect string value: '\xD0\x9B\xD1\x8E\xD0\xB1...' for column 'Link' at row 1242
cursor.executemany(sql, variables[i])
mergeDB.py:85: Warning: Incorrect string value: '\xD0\xA2\xD0\xB0\xD0\xB9...' for column 'Link' at row 1310
cursor.executemany(sql, variables[i])
mergeDB.py:85: Warning: Incorrect string value: '\xD0\x9B\xD1\x8E\xD0\xB1...' for column 'Link' at row 1468
cursor.executemany(sql, variables[i])
mergeDB.py:85: Warning: Incorrect string value: '\xD0\x94\xD0\xBD\xD0\xB5...' for column 'Link' at row 1687
cursor.executemany(sql, variables[i])
mergeDB.py:85: Warning: Incorrect string value: '\xD0\x91\xD0\xB5\xD1\x81...' for column 'Link' at row 1848
cursor.executemany(sql, variables[i])

From previous SOF threads, I have tried using inside my python file:

ALTER TABLE dbx.tablex MODIFY Link VARCHAR(255) CHARACTER SET utf8; COLLATE utf8_general_ci NOT NULL;

but this gives me an error:

ALTER TABLE dbx.tablex MODIFY Link VARCHAR(255) CHARACTER SET utf8; COLLATE utf8_general_ci NOT NULL;
          ^
SyntaxError: invalid syntax

How does one solve this problem? How can I simply delete (tell python not to load the rows) the rows which have the “Incorrect String Value?” within this loop:

for i in range(0,n):
    cursor.executemany(sql, variables[i])

Answer 1

ALTER TABLE dbx.tablex MODIFY Link VARCHAR(255) CHARACTER SET utf8; COLLATE utf8_general_ci NOT NULL;

This is a MySQL query, not a Python statement. You need to pass the above to MySQL.

Also, [] * 10 does not do what you think it does. And the proper way to iterate over a list of variables is:

for variable in variables:
    cursor.execute(sql, variable)

But you can do this directly:

cursor.executemany(sql, variables)