We have seperate Java instances program processing raw weblog like that:
jvm instance 1 read fileA --> process
jvm instance 2 read fileA ---> process
jvm instance 3 read fileA ---> process
....
I think when the number of jvm instance increase , the number of disk IO process ‘ll increase. And there comes a time this solution is can not work propertly.
So can anyone tell me another solution for reduce the disk IO.
I think an ideal is using a JMS server (like Apache ActiveMQ) to read file an store in queue and process.
have any problem if i use JMS ???
Please kindly help me.
Event driven solution is certainly a good option here, so JMS would probably be a good solution.
But you should keep in mind that if your consumers won’t keep up with producer and you’ll be using persistent delivery, messages will be stored on your hard drive and this will cause disk IO. But I think this won’t be a problem, as you can always increase number of concurrent consumers, or even use cluster (which is really easy to configure with ActiveMQ for example) to keep up with load.
To summarize, I think that JMS would be a great solution to your problem, as you won’t need to actively poll filesystem for changes and makes it really easy to scale your processing application.
If you are interested in topic of integration you might visit enterprise integration site and read a extremely good book by Gregor Hohpe and Bobby Woolf on this topic. You can find link to it at the mentioned site. In it you’ll find all pros and conses of both approaches as well as familiarize yourself with others available. Anyways messaging is definitely great way to go.
You might consider using camel framework as an implementation of mentioned there patterns.