Sign Up to our social questions and Answers Engine to ask questions, answer people’s questions, and connect with other people.
Login to our social questions & Answers Engine to ask questions answer people’s questions & connect with other people.
Lost your password? Please enter your email address. You will receive a link and will create a new password via email.
Please briefly explain why you feel this question should be reported.
Please briefly explain why you feel this answer should be reported.
Please briefly explain why you feel this user should be reported.
For all the ETLs I have written so far, I have never made them transactions – i.e. if table 4 fails, roll everything back.
What is the best practice in this regard?
To “BeginTran + Commit” or not to “BeginTran + Commit”
EDIT: I have one master package calling 4 other packages – is it possible to roll them all up into one transaction?
begin+commit in manageable batch sizes. You don’t want to wrap a 6 hours import into a single transaction every night. Keep your batches at a size that can finish in 2-3 minutes at most. That you will hit data purity issues that will fail an ETL is a given, so at least reduce the impact to something manageable (ie. don’t trigger a rollback that will last another 6 hours to complete).