Question

I am trying to import a large text file into a MySQL database. The SQL statement is as follows:

LOAD DATA INFILE '/tmp/epf/full/album_popularity_per_genre' 
INTO TABLE album_popularity_per_genre 
CHARACTER SET UTF8 FIELDS TERMINATED BY X'01' LINES TERMINATED BY '\n' 
IGNORE 45 LINES (export_date, storefront_id, genre_id, album_id, album_rank)

The above works when I run it in phpMyAdmin, however when I write a simple function in Python that uses the above SQL statement I get an error.

Here is the Python code,

def test():
    dbConnection = MySQLdb.connect(
    charset='utf8', 
    host='localhost', 
    user='root', 
    passwd='root', 
    db='epf')

    cursor = dbConnection.cursor()

    exStr = """LOAD DATA INFILE '/tmp/epf/full/album_popularity_per_genre' 
               INTO TABLE album_popularity_per_genre CHARACTER SET UTF8 
               FIELDS TERMINATED BY X'01' LINES TERMINATED BY '\n' 
               IGNORE 45 LINES 
               (export_date, storefront_id, genre_id, album_id, album_rank)"""

    try:
        cursor.execute(exStr)
    except MySQLdb.Warning, e:
        print "Warning %s" % (str(e))
    except MySQLdb.IntegrityError, e:     
        print "Error %d: %s" % (e.args[0], e.args[1])

    #Clean up
    cursor.close()
    dbConnection.close()

The error I get is as follows,

Warning Data truncated for column 'album_rank' at row 1

My question now is, why does the raw SQL statement work but when I try to run the Python code, no data is imported into the database?

Answer 1

The Python DBAPI is implicitly transactional. Try adding dbConnection.commit() after the execute.