Question

I working on app what should get list of users from db and update thie details from directory (ldap or AD). I what do this procedure on multicore machine so I created this app (code below). I’m using CompletionService and getting the results in Future object.

After some time, I’m getting out of memory error with ‘unable to create new native thread’ message. In task manager I see that app created enormous amount of threads but I asked create fixed thread pool with size equals amount of my processors.

What is wrong with my code?

class CheckGroupMembership {
public static void main(String[] args) throws Exception {

    final ExecutorService executor = Executors.newFixedThreadPool(**Runtime.getRuntime().availableProcessors()**);

    CompletionService<LdapPerson> completionService =
        new ExecutorCompletionService(executor)<LdapPerson>(executor);

    final int limit = 2000;

    DocumentService service1 = new DocumentService();
    List<String> userNamesList = service1.getUsersListFromDB(limit);

    List<LdapPerson> ldapPersonList = new ArrayList() <LdapPerson> (userNamesList.size());
    LdapPerson person;

    for (String userName : userNamesList) {
        completionService.submit(new GetUsersDLTask(userName));
    }

    try {
        for (int i = 0, n = userNamesList.size(); i < n; i++) {
            Future<LdapPerson> f = completionService.take();
            person = f.get();
            ldapPersonList.add(person);
        }
    } catch (InterruptedException e) {

        System.out.println("InterruptedException error:" + e.getMessage());
    } catch (Exception e) {
        System.out.println(e.getMessage());
    }
    System.exit(0);
}
}

ERROR CheckGroupMembership:85 – java.lang.OutOfMemoryError: unable to create new native thread
java.util.concurrent.ExecutionException: java.lang.OutOfMemoryError: unable to create new native thread
at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
at java.util.concurrent.FutureTask.get(FutureTask.java:83

GetuserDLs task

public class GetUsersDLTask implements Callable<LdapPerson> {
private String userName;

public GetUsersDLTask(String u) {
    this.userName = u;
}

@Override
public LdapPerson call() throws Exception {
    LdapService service = new LdapService();
    return service.getUsersDLs(userName);
}

}

Answer 1

I am having a hard time believing you are not creating a Thread in GetUsersDLTask (or at least it’s service Object). If you look at your stacktrace, the Exception is being thrown from the Future’s get() method. The only way this exception get’s set is after the Executor invokes Callabale.call(). Any throwable that occurs in the call() method will be set in the Future’s internal exception field

For example:

Thread Pool: 
    Thread-1
      invoke call()
        call() 
          Create Thread
            throw OutOfMemoryError 
         propogate error to Thread pool
      set exception

Otherwise, this exception would be occurring when you submit the ask to thread pool, not when you get from the future.