Question

I need to run a couple of relatively simple SQL update statements to update a single column in an Oracle table with 14.4 million rows. One statement runs a function written in Java and the JVM runs out of memory as I’m doing the update on all 14.4 million rows.

Have you written a kind of batch PL/SQL routine that can break this simple update into sets of, say, 10K records per batch? I know that if I can commit my updates after a bunch of records, it’ll go a lot faster and I won’t run out of memory. I’m sure there’s a simple way to do this using a FOR loop and row_num but I’m not making much progress.

Here are the two statements I need to run for each batch of n records:

first one:

 update vr_location l set l.usps_address=(
   select mylib.string_utils.remove_duplicate_whitespace(
   house_number || ' ' || pre_street_direction || ' ' || street_name || ' ' || 
   street_description || ' ' || post_street_direction)
 from vr_address a where a.address_pk=l.address_pk);

second:

update vr_location set usps_address = mylib.usaddress_utils.parse_address(usps_address);

Answer 1

Do an initial select to retrieve some kind of grouping attribute, so that you end up with groups that have the desired number of rows. Experiment with the grouping clause, for instance the last three digits of a zip-code or something semi random.

Loop over the grouping clause, using the clause as parameter to limit the rows targeted by each update statement. commit at the end of each iteration.