Question

My task is to write a stored procedure that will first validate the data from a temporary table and then insert the data into the main table.
For this I am planning to iterate over each row of the temp table, validate it using some other stored procedure or user defined function and then insert the data into the main table.

My problem is how to iterate over the rows of temp table without using CURSORS because they are very slow and memory consuming. I want to use some looping structure instead of CURSOR.

Of course if any one has any other algorithm for the above problem there suggestions are welcome.

PS: I am using MYSQL DB

Answer 1

Without the use of Cursor, you could iterate using a temporary table and a While..Do statement.

Let’s say you have two tables

CREATE TABLE `user` (
  `id` int(11) NOT NULL auto_increment,
  `name` varchar(45) NOT NULL,
  PRIMARY KEY  (`id`)
) ENGINE=MyISAM;

And

CREATE TABLE `tmp_user` (
  `id` int(11) NOT NULL auto_increment,
  `name` varchar(45) NOT NULL,
  PRIMARY KEY  (`id`)
) ENGINE=MyISAM;

Create the following routine, and adjust the validation process:

DELIMITER $$
USE `routines_sample`$$
CREATE PROCEDURE `nocursor`()
BEGIN
  Declare validId int;
  Declare validName varchar(45);

  -- drop the temporary table if exists
  Drop table if exists `routines_sample`.tmp_validation;
  -- create the temporary table, here you could use Engine=Memory
  Create table `routines_sample`.tmp_validation (`id` int not null, `name` varchar(45) not null, `valid` bit(1) not null) Engine=MyISAM;

  -- insert into the temporary table with a valid flag = 0 (false)    
  Insert into `routines_sample`.tmp_validation (`id`, `name`, `valid`)
  Select tu.id, tu.name, 0
  From `routines_sample`.tmp_user tu;

  -- while exists data to validate on temporary table do something    
  While exists(Select `id` From `tmp_validation` Where `valid` = 0) Do

    Select `id`, `name` Into @validId, @validName From tmp_validation Where `valid` = 0 Limit 1;

    -- do your validation
    Select @validId, @validName;

    -- don't forget to update your validation table, otherwise you get an endless loop    
    Update `tmp_validation` 
    Set `valid` = 1
    Where `id` = @validId;

  END WHILE;

  -- insert only valid rows to your destination table    
  Insert into `routines_sample`.`user` (`name`)
  Select `name` From `tmp_validation`
  Where `valid` = 1;

  -- drop the temporary table    
  DROP TABLE tmp_validation;

END$$

DELIMITER ;