Question

We have a lengthy ETL process that flows data from input files through a series of tables.

I am considerin the unlikely possibility of adding relational integrity to our tables but I do not want our ETL process to die when it encounters a violation. I also want the records that violate the referential integrity to still be loaded. However, in the end, I want to be aware of all of the violations of referential integrity.

Method 1:
I could leave referential integrity off and write a SQL procedure that runs a bunch of stored procedures to identify records that violate relational integrity, but I really like the idea of the relation integrity being on the tables themselves because i feel that this documents the database in the best place – the db.

Method 2:
Instead of write a set of custom queries to identify violators, I am thinking that we should drop all ref integrity in the beginning of our process and then add it at the end. Where we get exceptions, we know there are violations. I kind of like this approach, but uunlike Method 1 where a SQL can be written to target only the records just added for potential violators, adding ref integrity back will probably reexamine the whole table — a table that is ever growing. When ref integrity is turned back on, the consumer of the data can be assured that the data is “good” without peforming any more on the fly queries. I like that…

Is there a third approach?
I see T-SQL supports commands like

NOCHECK CONSTRAINT  
ON UPDATE  NO ACTION  
ON INSERT  NO ACTION

but I am not sure how they are really intended to be used. For example,

ALTER TABLE dbo.TableName
NOCHECK CONSTRAINT FK01

Is the intent of this to turn off constrationmt checking when you have a reliable source? I assume that if it is off and then turned on in this manner that the setting chg only applies to future operations.

What best approach would you use to allow a process to complete to the end and still identify all of the relational integrity or would-be relational integrity violations?

Answer 1

1.I found NO ACTION name is a bit misleading because it means DML will fail if it violates constraint. Some RDMS, mysql in particular, have a better keyword – RESTRICT which is more descriptive.
2. You can temporary disable/enable all constraints with ALTER TABLE ... NOCHECK/CHECK CONSTRAINT ALL