Question

I’m building a .NET application that talks to an Oracle 11g database. I am trying to take data from Excel files provided by a third party and upsert (UPDATE record if exists, INSERT if not), but am having some trouble with performance.

These Excel files are to replace tariff codes and descriptions, so there are a couple thousand records in each file.

|   Tariff   |        Description        |
|----------------------------------------|
| 1234567890 | 'Sample description here' |

I did some research on bulk inserting, and even wrote a function that opens a transaction in the application, executes a bunch of UPDATE or INSERT statements, then commits. Unfortunately, that takes a long time and prolongs the session between the application and the database.

public void UpsertMultipleRecords(string[] updates, string[] inserts) {
    OleDbConnection conn = new OleDbConnection("connection string here");
    conn.Open();
    OleDbTransaction trans = conn.BeginTransaction();
    try {
        for (int i = 0; i < updates.Length; i++) {
            OleDbCommand cmd = new OleDbCommand(updates[i], conn);
            cmd.Transaction = trans;
            int count = cmd.ExecuteNonQuery();
            if (count < 1) {
                cmd = new OleDbCommand(inserts[i], conn);
                cmd.Transaction = trans;
            }
        }
        trans.Commit();
    } catch (OleDbException ex) {
        trans.Rollback();
    } finally {
        conn.Close();
    }
}

I found via Ask Tom that an efficient way of doing something like this is using an Oracle MERGE statement, implemented in 9i. From what I understand, this is only possible using two existing tables in Oracle. I’ve tried but don’t understand temporary tables or if that’s possible. If I create a new table that just holds my data when I MERGE, I still need a solid way of bulk inserting.

Answer 1

The way I usually upload my files to merge, is by first inserting into a load table with sql*loader and then executing a merge statement from the load table into the target table.

A temporary table will only retain it’s contents for the duration of the session. I expect sql*loader to end the session upon completion, so better use a normal table that you truncate after the merge.

merge into target_table t
using load_table l on (t.key = l.key) -- brackets are mandatory
when matched then update
set t.col = l.col
,   t.col2 = l.col2
,   t.col3 = l.col3
when not matched then insert
(t.key, t.col, t.col2, t.col3)
values
(l.key, l.col, l.col2, l.col3)