Question

I am having an application for handling more than 10000000 data.

The MainTable has more than 10000000 data

I am trying to Insert the Data into a SubTable From the Main Table as

INSERT INTO SubTable(Value1,Value2)
SELECT Value1,Value2 FROM MainTable
GROUP BY Value1_ID;

After performing certain processing in SubTable..Again I update the new values into the Main Table as

UPDATE MainTable inf,SubTable in
SET inf.Value1=in.Value1, inf.Value2=in.Value2
WHERE inf.Value1_ID= in.Value1_ID;

While Running this query the Entire Server gets very slow and it stops the entire other transaction.I am using the JDBC Driver Manager connection here. How to avoid this? How to solve this problem?

Answer 1

If it’s something that you have to do only once in a while, instead of updating the whole table in a single update, you can set up a small script that will update by batch of rows every few seconds/minutes or so. The other processes will have their query executed freely between two updates.
For example, by updating a batch of 100,000 rows every minutes, if your tables have the right indexes, that would take 1~2 hours, but with a far lesser impact on the performance.

The other solution would be do the update when the activity on the server is at its lowest (maybe during the week-ends?), that way you won’t impact the other processes as much.