Question

We have a table EVAPP_INTERFACE that did not specify a scale or precision on a number of numeric columns. We need to modify the column definition to add appropriate scale and precision but we cannot change the order of the columns in the table. In order to accomplish this, we are doing the following

• We copy all the data to a new table EVI
• We set the numeric columns in the existing EVAPP_INTERFACE table to NULL
• We change the precision of the EVAPP_INTERFACE columns
• We copy the data back

When the data is copied back, however, some rows generate an exception because they exceed the new scale and/or precision settings (i.e. a value of 1 billion in a column that is supposed to store an interest rate).

ERROR : ORA-01438: value larger than specified precision allowed for this column 

I want to identify which column and which rows have this bad data.

First the duplicate table

 select count(*) into countTab from USER_TAB_COLUMNS where TABLE_NAME = 'EVI'; 
IF  (countTab <> 0) then 
    execute immediate 'drop table EVI';
    execute immediate 'create table EVI as (select * from EVAPP_INTERFACE)'; 
ELSE
    execute immediate 'create table EVI as (select * from EVAPP_INTERFACE)'; 
END IF;
execute immediate 'TRUNCATE TABLE EVAPP_INTERFACE';

Then, I change precisions: There are 226 such blocks throughtout the script.

 select count(*) into countCol from USER_TAB_COLUMNS where TABLE_NAME = 'EVAPP_INTERFACE' and COLUMN_NAME = 'PST_NUM' and DATA_SCALE is null; 
  IF    (countCol <> 0) then   
 execute immediate 'alter table EVAPP_INTERFACE modify PST_NUM NUMBER(14,2)' ; 
     DBMS_OUTPUT.put_line('  EVAPP_INTERFACE.PST_NUM has been modified to the required precision'); 
END IF; 

Then, I insert back the old data. This is where it bombs

 execute immediate 'INSERT INTO EVAPP_INTERFACE SELECT * from EVI';

So, I am not entirely sure if SQLERRM and SQLCODE will give the full stacktrace. And, I don’t have enough dataset to verify this. Can anyone confirm if there is anyway that I can find out which particular column is causing the issue? I have already provided NUMBER(14,2) as the precision for most columns, and if possible I want to erase the bad data instead of increasing precision. And since it is production data, I am not allowed to import the data to inspect it.

Answer 1

You probably want to use DML Error Logging to log all the rows that have problematic data.

I’m not sure why you’re using dynamic SQL everywhere– it would make much more conventional to create the table outside of the script rather than dropping & recreating it every time. That would allow you to use static SQL to refer to the table and would let you move lots of error checking into the compilation phase rather than the execution phase. I’ll assume, though, that you have solid reasons for wanting this to be entirely dynamic.

You would need to create an error table (and you’d need to drop the error table if you’re going to drop the base table so that you can recreate it). And you would need to modify your INSERT to add a LOG ERRORS INTO. Something like

dbms_errlog.create_error_table( 'EVAPP_INTERFACE', 
                                'EVAPP_ERROR' );

to create the error table and

EXECUTE IMMEDIATE 
  'INSERT INTO evapp_interface ' ||
  '  SELECT * ' ||
  '    FROM evi ' || 
  '  LOG ERRORS INTO evapp_error ' ||
  '  REJECT LIMIT UNLIMITED';

to insert the data into EVAPP_INTERFACE while writing any failures to EVAPP_ERROR (the error table we created in the step above).