Question

This is my first Stack Overflow post! The community has been a great help!

I’ve got a 90K row CSV I’m trying to insert into a table, via a temp table with CSV like:

CREATE TEMPORARY TABLE temp_product like product;
LOAD DATA LOCAL INFILE "myfile.csv"
...

But I am trying to wrap up the insert and a few additional operations via a procedure, which contains a transaction, and an exit handler incase anything fails, or throws a warning:

CREATE PROCEDURE update_products ()
BEGIN
    DECLARE EXIT HANDLER FOR SQLEXCEPTION, NOT FOUND, SQLWARNING ROLLBACK;
    START TRANSACTION;
    ....

I like this approach because it allows me to automatically rollback if any errors are encountered, and also wraps up the whole logical chunk of transactions nicely. Alas:

Error : LOAD DATA is not allowed in stored procedures

(I’m not certain why this is? I guess it’s just not supported yet?) I’m not certain I have the syntax correct (or if this is even possible) but I would even be OK with something like:

DECLARE EXIT HANDLER FOR SQLEXCEPTION, NOT FOUND, SQLWARNING ROLLBACK;
START TRANSACTION;

But that throws the obvious syntax error:

Error : You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'DECLARE EXIT HANDLER FOR SQLEXCEPTION, NOT FOUND, SQLWARNING ROLLBACK' at line 1

It seems I could make this work if I could load the CSV in a different manner, but I’d likely lose the speed of doing the inserts in bulk, and/or have to use a disk-based table as I would likely need to make multiple connections to it. As far as I understand, a temporary table exists only for the single connection/transaction, and I’m not 100% certain how my MySQL client, navicat, manages this.

I’m also not against wrapping this all in PHP if that’s the most sane way to manage a script that has automatic rollback upon error/warning in transactions with LOAD DATA.

Feel free to tell me I am approaching this totally wrong as well. I feel like there’s probably a much smarter way to do what I’m doing.

Answer 1

Unfortunately there is no support for LOAD DATA INFILE in stored procedures. I do recall once seeing a workaround hack where someone used MySQL stored procedure to call a shell script to actually perform the upload, but that seems like a potentially problematic solution, security-wise. You might be able to Google it this solution.

The best bet would probably be to use some sort of script (PHP, shell, etc.) to perform the load into the temporary table and then call the stored procedure to actually populate into the permanent table. I would think that as long as you had transactional support for this piece, you can roll back the part that really matters (populating the permanent table).