Question

I am designing a server application, that is supposed to crunch a lot of data continuously and present results on demand using web interface.

The operating scheme goes roughly like this:

1. An electronic sensor array constantly spills data into ramdisk through USB
2. A “flusher” application processes data as fast as it can and loads it into db (staging area)
3. Using triggers, db performs calculations on data and stores results in another schema (data area)
4. Client webapp can display processed data in graphs/reports etc. on demand

The solution would ideally look like this:

1. Database server – PostgreSQL
2. Have an administration web interface, that can monitor the flusher (i.e. records processed per hour or something like that) and if implemented as separate daemon, control it.
3. Flusher and Client applications written in Java, ideally using J2EE

Now the problem that keeps bugging me and I can’t find the answer: How to go about writing the flusher component, i.e. a process that constantly runs in background in J2EE.

By scouring the web, basically three possibilities emerged:

a) Write the flusher as message driven bean and control it from master application using JMS. However: I don’t like the idea of having a MDB running constantly, I’m not even sure that that’s possible

b) Write the flusher as EJB and control it using Timer/Scheduling service. However: the events are not really timed, it just needs to run in infinite loop until told not to do so, just seems wrong usage of the technology.

c) Write the flusher as separate java application, run it as OS service (Linux or Windows) and control using startup scripts through ProcessBuilder invoked from EJB. To monitor it’s status, use JMS. However: this just seems to me as overly complicated solution, platform dependent and maybe even unreliable and as EJB should not spawn/manage it’s own threads, which ProcessBuilder basically does, it just seem wrong.

Basically, none of these look right to me and I cannot figure out, what would we the right solution in the Java/J2EE world.

Thank you
Thomas

Answer 1

I would write the “Flusher” app as a stand alone Java process. Perhaps use something like Java Service Wrapper to turn it into a service for your OS. I’m not very familiar with the options for interfacing with a RAM disk via Java, but you’re either going to end up with an InputStream which you can keep open for the life of the process and continually read from, or you’re going to continually poll from inside a while loop. It’s perfectly ok to do something like the following:

private volotile boolean stopFlag;

...

while(!stopFlag) {
  processNextInput();
}

Then you would have some other mechanism in another thread that could set stopFlag to true when you wanted to terminate the process.

As for monitoring the flusher JMX seems like a good solution. That’s exactly what it was intended for. You would create an MBean that would expose any kind of status or statistics you wanted and then other processes could connect to that MBean and query for that data.

The “Client” app would then be a simple servlet application which does reporting on your database and provides a pretty front end for the MBean from your flusher. Alternatively you could just monitor the flusher using a JMX console and not even involve the client with that piece of the system.

I don’t think EJBs really make sense for this system. I’m somewhat biased against EJBs, so take my advice with a grain of salt, but to me I don’t really see a need for them in this application.