Question

I am running some experiments using a Java program including Hibernate data storage. An Experiment can contain many Trials, and each Trial can have many Tests. Each Experiment is conducted in a single Spring transaction.

When there are small amount of Tests, everything is fine, but as data volumes increase I start to encounter Java heap space out of memory issues. The memory issues seem reasonable given the amount of data I am processing in a single transaction.

My current workaround for this is to:

Set up and save the initial Experiment
Run a set of Tests.
Summarise the Test results and create a Trial record
Delete the Test records
Save the completed Trial
Prompt the Garbage Collector to run

This works, and I can conduct extensive experiments using the framework. However, I no longer have access to perform data analysis on the detailed Test records, as I had to delete them to prevent memory issues.

What I would like to do is save the Test records to file, and then “unload” them from the heap. The Trial will still have a one-many with the saved Test records, but since the Tests are completed, I no longer need the details in memory.

Set up and save the initial Experiment
Run a set of Tests.
Summarise the Test results and create a Trial record
Save the Trial (with cascade saves for all Test records)
Unload the complete Test records 
Prompt the Garbage Collector to run

Can anyone think of a resolution for my problem ?

Answer 1

To prevent your objects to be be in the Hibernate Session (and be able to be garbage collected), you should simply detach your objects from Hibernates session (using session.evict()). In your case you should iterate through all Test instances after each save of a Trial, evict them, and then evict your Trial as well.

When you need your object again you can load your objects again using normal Hibernate methods (load or queries). You should probably consider lazy-loading on Trial.test if they are not needed.