Question

I am testing the fair scheduler mode for job assigning, however I get such error
java.io.IOException: Cannot run program “bash”: java.io.IOException: error=24, Too many open files

After a google, Most of them will tell to check how many files are currently open in the system (by unix command lsof) and how’s that number related to your system limit (check by bash command ulimit -n). Increasing maximum number of open files at a time is short-term solution in my opinion.

Is there anyway to avoid this?

Answer 1

The fact that your system is reaching the limit for #(max open files), you might have to check:

1. How many other operations are running on the system ?
2. Are they heavily opening many files ?
3. Is your hadoop job itself heavily opening many files ?
4. Is the current limit for #(max open files) too small on your system ? (you can google out the typical values). If its too small, consider increasing it

I think increasing the #(max open files) limit will work out. In long term you might again end up with that problem if #1,#2 and #3 are not addressed.