Question

I have written a java code which is a wrapper for combining 3 different softwares. I run the code for 100,000 times normally. At each run, different files should be opened, re-written and closed where all happen in a correct convention of try and catch. If I run the code on a Linux server, there would be no problem. However, when I run it on my mac after 10s of 1000, the Error of Too many open files occur and subsequently error in loading file X and Y and so on and the program terminates.

one more thing I would like to add: the code runs for 1000 times, then the setting gets changed and again for 1000 times and this process repeats. so for the first 4-5 times, there would be no problem, while after 6-7 times (means 6000 runs) this error occurs.

Answer 1

The most likely explanation is that you have a bug in your Java application that is causing it to leak file open files. The way to avoid the problem is to write your code so that file handles are always closed; e.g.

    // Using the 'try with resource' construct
    try (Reader r = new FileReader(...)) {
        // use the reader
    } catch (IOException ex) {
        // ...
    }

or

    // Using classic try/catch/finally
    Reader r = null;
    try {
        r = new FileReader(...);
        // use the reader
    } catch (IOException ex) {
        // ...
    } finally {
        if (r != null) {
            try {
                r.close();
            } catch (IOException ex) {
                // ignore
            }
        }
    }

The chances are that the program worked on one operating system and not another because the limit on the number of open files is different on the two systems. If the Java GC runs, it will close any unreachable file handles that it finds via the respective handle’s finalizer. However, this may not happen in time to prevent the “too many files open” exception.

---

If your application actually requires a large number of files to be open simultaneously, you will need to lift the limit. This is best done using the ulimit shell built-in, assuming that you are using an UNIX-based operating system.

Note however that there will still be a hard limit that it is not possible to exceed. And I believe that applies for Windows as well.