Question

I’m trying to do a procedure capable of converting broken UTF8 characters. This is my SP:

CREATE DEFINER=`root`@`localhost` PROCEDURE `conversorUTF8`()
BEGIN

DECLARE con_id, con_apellido, con_direccion VARCHAR (60);
DECLARE done INT DEFAULT 0;

DECLARE CID CURSOR FOR SELECT id, direccion FROM alumnos;
DECLARE CONTINUE HANDLER FOR NOT FOUND SET done = 1;
DECLARE exit handler for sqlexception rollback; 
DECLARE exit handler for sqlwarning rollback;


OPEN CID;

read_loop: LOOP
    FETCH CID into con_id, con_direccion;
    IF done THEN 
        LEAVE read_loop;
    END IF;

    CASE WHEN (SELECT CONVERT(CONVERT(CONVERT(con_direccion USING latin1)using binary)using utf8)) IS NOT NULL THEN
        UPDATE alumnos SET direccion=CONVERT(CONVERT(CONVERT(con_direccion USING latin1)using binary)using utf8) where id = con_id ;

    END CASE;
END LOOP;

CLOSE CID;

END

The problem is: when I already have a string converted with a ‘á’, ‘é’ etc, and I try to convert that string again, the update fail with this error:

UPDATE alumnos SET direccion=CONVERT(CONVERT(CONVERT(con_direccion USING latin1)using binary)using utf8) where id = con_id ; 

Invalid utf8 character string.

That’s why my CASE STATEMENT:

CASE WHEN (SELECT CONVERT(CONVERT(CONVERT(con_direccion USING latin1)using binary)using utf8)) IS NOT NULL THEN

But something is still failing.

Any tips?

Here the solution :

alter table alumnos change direccion direccion VARCHAR(100) CHARACTER SET latin1;
alter table alumnos change direccion direccion VARBINARY(100);
alter table alumnos change direccion direccion VARCHAR(100) CHARACTER SET utf8;

Thanks 🙂

Answer 1

I might be reading this wrong, but… if your columns are utf8 strings which were stored as latin1 (and typically swedish, at that) column, and you altered your table so that the column used a new charset and new collation rules, then either:

1. You have not altered the table in production yet…

   In this case hop straight to the WP documentation on how to do it.

2. The altered table is already in production…

   In this case you’re going to convert only part of your table — the part before the alter occurred. Failing to do so will likely get you error messages like the one you’re getting when converting non-ascii characters back and forth. (You might be able to detect strings with broken utf8 using a couple of regular expressions.)