Question

I’m trying to insert rows using raw sql in django:

The input file has 15 lines that are “commented out” using ‘#’.

The rest of the file is tab delimited and terminated by ‘\n’.

I am trying to ignore the first 15 lines using the IGNORE <#> LINES clause in my LOAD DATA INFILE statement.

using the following code:

def my_custom_sql():
    from django.db import connection, transaction
    cursor = connection.cursor()
    statement = "LOAD DATA LOCAL INFILE '/home/dev/ttam/data/temp.ttam' INTO TABLE ttam_genotypeentry (snp_id, @dummy1, @dummy2, genotype) SET subject_id=%i IGNORE 15 LINES;" % subject.pk
    cursor.execute(statement)
    transaction.commit_unless_managed()

i get the following error:

DatabaseError: (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'IGNORE 15 LINES' at line 1")

With the statement (removing the IGNORE 15 LINES):

statement = "LOAD DATA LOCAL INFILE '/home/dev/ttam/data/temp.ttam' INTO TABLE ttam_genotypeentry (snp_id, @dummy1, @dummy2, genotype) SET subject_id=%i;" % subject.pk

I get this error:

IntegrityError: (1452, 'Cannot add or update a child row: a foreign key constraint fails (`ttam_genotypeentry`, CONSTRAINT `snp_id_refs_rs_id_50bcd16b` FOREIGN KEY (`snp_id`) REFERENCES `ttam_snp` (`rs_id`))')

If i manually remove the first 15 lines of the input file, the operation continues without issue.

I am using:

django: Django version 1.3 rc 1

mysql: Server version: 5.1.49-1ubuntu8.1 (Ubuntu)

How do i ignore the first 15 lines of my input file?

Answer 1

Try moving the IGNORE n LINES clause to before the SET clause.

statement = "LOAD DATA LOCAL INFILE '/home/dev/ttam/data/temp.ttam' INTO TABLE ttam_genotypeentry (snp_id, @dummy1, @dummy2, genotype) IGNORE 15 LINES SET subject_id=%i;" % subject.pk