Home/ Questions/Q 8534213
In Process

The Archive Base Latest Questions

Editorial Team
  • 0
Asked: 2026-06-11T10:08:32+00:00

I have bunch of tools stored in one table and each tools associated transactions

  • 0

I have bunch of tools stored in one table and each tools associated transactions stored in its child table. I have an algorithm that determines what tool to buy based on its performance in the transaction table entries. My output of the algorithm may recommend to buy a tool and may not based on its performance.

My end goal is only to show the list of tools page by page ( with server side pagination ) which are recommended to buy by the algorithm. My question here is, if I fetch first 10 tools and its transactions and give as input to the algorithm, my algorithm may or may not recommend all the tools to buy, in such a case, the tools which are eligible to display are not enough for the current page. I have to again go back to the data base and fetch few more until my results are enough to display the current page ( which includes too many db iterations ).

I am currently fetching everything and caching the results collection at the serverside ( with Ehcache) and applying the server side pagination over my collection. But this is pushing the server to go out of memory if too many concurrent users hit this page.

What is the best solution for this kind of issue..!!?

Share

Leave an answer

You must login to add an answer.

Need An Account,

1 Answer

  1. Editorial Team

    I’m afraid there is no silver bullet solution, but I would like to share my thoughts here

    Usually if you want an accurate result for the pagination you ideally need to run all the iteration in the same transaction, otherwise the data just won’t be consistent because of parallel updates that can be done to your persistent storage. However I’m not sure its possible from your post.

    So, consider the following:

    • Don’t store everything in memory like you do now, instead maintain some share objects pool in memory and the parallel customers would only store pointers to your objects instead of the whole objects.

    • Use some kind of serialization of your results. Besides trivial serialization you may be interested in “off heap memory” it works with ehcache : http://ehcache.org/documentation/user-guide/bigmemory

    • Consider to store only tool id accross the iteration and maybe bare minimum of data that algorithm needs to make a decision. Then when you’re ready to display results make additional request and bring from the persistence layer what you need for showing up the results by supplying series of ids as an input. The chances are that you still won’t need all the data and this will save up your memory

    • Increase the heap. Trivial 🙂

    Hope this helps

    • 0