Sign Up to our social questions and Answers Engine to ask questions, answer people’s questions, and connect with other people.
Login to our social questions & Answers Engine to ask questions answer people’s questions & connect with other people.
Lost your password? Please enter your email address. You will receive a link and will create a new password via email.
Please briefly explain why you feel this question should be reported.
Please briefly explain why you feel this answer should be reported.
Please briefly explain why you feel this user should be reported.
We are putting together a platform that can supposedly load any jar file and run statistical models. The issue im facing right now is that some models run too big to fit on our platform causing heap out of memory errors. I know there has been research done on this but I cant find them anymore. In essence, how does google app engine do this? Has anybody heard of any disk based heap space?
I assume that you’ve already expanded your heap as much as is realistically possible (given the constraints of your physical memory, processor architecture, JVM and OS).
Beyond, that the answer is that no JVMs that I’m aware of implement disk based heap space. However, it is not a totally ludicrous idea. There once was a community of academics (mostly) working on the problem of “orthogonal persistence” where objects migrate transparently between memory and a persistent store. Unfortunately, there were some fundamental issues that made this technology … hard. (For instance, garbage collecting the persistent store, getting logically consistent checkpoints with multi-threaded applications, and dealing with code change.) Anyway, the upshot is that field of research has gone quiet.