Question

I am trying to read a number of files in a folder and process them simultaneously using threads.

The structure of the program is like this :

// Assuming there are 5 files in the directory

// creating the threads
ExecutorService pool = Executors.newFixedThreadPool(5)
ExecutorCompletionService service = new ExecutorCompletionService(pool)

directory.listFiles().each { eachFile ->
   service.submit(new FileReader(eachFile, param2))
}


// the FileReader class
class FileReader implements Callable {
    File file
    String param
    FileReader(File file, String param){
        this.file = file
        this.param = param
    }

   Object call(){
      LOG.info("Processing file" + filePath)
      ConfigInfo configInfo = new ConfigInfo()
  configInfo.setFilePath(filePath);
  configInfo.setReaderUid(readerUid);
  configInfo.setPatternsMap(patternsMap);
  new LogfileDataProcessor(configObject, param).processFileContent()
   }

}

The call method here creates another object and calls a method on it.

But oddly enough the program terminates after executing some lines in the call method(it does not reach the final statement in it). I am confused here. Can someone throw some light on what is happening. Please help me

Answer 1

You need your program wait for thread to be finished. You can use CountDounLatch for example for this:

CountDownLatch latch = new CountDownLatch(numberOfFilesInDirectory);

directory.listFiles().each { eachFile ->
   service.submit(new FileReader(eachFile, param2))
}

latch.await();


// And in your Callable:
class FileReader implements Callable {
File file
String param
FileReader(File file, String param){
    this.file = file
    this.param = param
}

public Object call() {
    try {
    LOG.info("Processing file" + filePath)
    ConfigInfo configInfo = new ConfigInfo()
    configInfo.setFilePath(filePath);
    configInfo.setReaderUid(readerUid);
    configInfo.setPatternsMap(patternsMap);
    new LogfileDataProcessor(configObject, param).processFileContent();
    } finally {
        latch .countDown();
    }
}

You can pass latch into thread as constructor arguments.