Question

We recently fixed an issue with the character encoding being read incorrectly into our system from the text files by making sure the file is UTF-8 and the Java code opens these files in UTF-8 encoding.

However, we had ended up adding a lot of records across the entire database tables with incorrect characters being inserted i.e. °F was read as �F. So even though we have fixed this now, we need to clean up the database tables now to rectify this anomaly.

Can anyone please suggest me ways to achieve this?

Answer 1

I had a similar problem a while back. Luckily, the number of columns that it affected was limited to a small number, and those columns had the same name throughout the database.

I solved this by writing a script that does the following:

1. disable foreign key constraints
2. build up a list of tables which contain the
   target columns
3. update all the tables in your list using a
   REGEXP_REPLACE
4. commit the data re-enable the constraints

This used a healthy dose of dynamic SQL, pulling data from the user_constraints and user_tab_columns, filtering on the specific column names I was targeting.

Here’s a rough skeleton to get you started, I’ve just thrown it together quickly, so it isn’t tested. Also, if you have triggers to worry about, you’ll need to disable those too:

-- disable constraints
BEGIN
    FOR c IN (
        SELECT c.owner, c.table_name, c.constraint_name, c.constraint_type
        FROM user_constraints c
        INNER JOIN user_tables t ON (t.table_name = c.table_name)
        AND c.status = 'ENABLED'
        AND c.constraint_type NOT IN ('C', 'P')
        ORDER BY c.constraint_type DESC
    )
    LOOP
        dbms_utility.exec_ddl_statement('alter table '||c.table_name||' disable constraint ' || c.constraint_name);
    END LOOP;
END;

-- do the updates
BEGIN
    FOR t IN (
        SELECT table_name, column_name
        FROM user_tab_columns
        WHERE column_name = 'TEMPERATURE'
        AND data_type = 'VARCHAR2';
    )
    LOOP
        dbms_utility.exec_ddl_statement('UPDATE '||t.table_name||' SET ' ||t.column_name||' = '||''GOOD VALUE''||' WHERE '||t.column_name||' = '||''BAD VALUE'');
    END LOOP;
END;

-- re-enable constraints
BEGIN
    FOR c IN (
        SELECT c.owner, c.table_name, c.constraint_name, c.constraint_type
        FROM user_constraints c
        INNER JOIN user_tables t ON (t.table_name = c.table_name)
        AND c.status = 'DISABLED'
        AND c.constraint_type NOT IN ('C', 'P')
        ORDER BY c.constraint_type ASC
    )
    LOOP
        dbms_utility.exec_ddl_statement('alter table '||c.table_name||' enable constraint ' || c.constraint_name);
    END LOOP;
END;
/