Question

The workaround for an issue in supervisord is to:
“compile a Python that supports > 1024 file descriptors”

https://github.com/Supervisor/supervisor/issues/26

Can someone please walk me through what changes are necessary to accomplish this? I have the python 2.7.2 source extracted and ready to go.

Running centos 5.6, if that matters.

Thanks.

Update: ulimit -n is already set to 65535. This is the full error I’m getting when starting supervisord:

Traceback (most recent call last):
File “/usr/local/bin/supervisord”, line 8, in
load_entry_point(‘supervisor==3.0a10’, ‘console_scripts’, ‘supervisord’)()
File “/usr/local/lib/python2.7/site-packages/supervisor-3.0a10-py2.7.egg/supervisor/supervisord.py”, line 372, in main
go(options)
File “/usr/local/lib/python2.7/site-packages/supervisor-3.0a10-py2.7.egg/supervisor/supervisord.py”, line 382, in go d.main()
File “/usr/local/lib/python2.7/site-packages/supervisor-3.0a10-py2.7.egg/supervisor/supervisord.py”, line 95, in main
self.run()
File “/usr/local/lib/python2.7/site-packages/supervisor-3.0a10-py2.7.egg/supervisor/supervisord.py”, line 112, in run
self.runforever()
File “/usr/local/lib/python2.7/site-packages/supervisor-3.0a10-py2.7.egg/supervisor/supervisord.py”, line 230, in runforever
r, w, x = self.options.select(r, w, x, timeout)
File “/usr/local/lib/python2.7/site-packages/supervisor-3.0a10-py2.7.egg/supervisor/options.py”, line 1113, in select
return select.select(r, w, x, timeout)
ValueError: filedescriptor out of range in select()

Answer 1

That’s actually the limit of the underlying select(2) system call.

From the man page:

An fd_set is a fixed size buffer.  Executing FD_CLR() or FD_SET() with a value of fd 
that  is  negative  or  is equal  to  or  larger  than  FD_SETSIZE will result in 
undefined behavior. 

And the standard FD_SETSIZE is 1024.

/usr/include/linux/posix_types.h:#define __FD_SETSIZE   1024

So it’s not a Python issue. The poll(2) and epoll(2) system calls have a much larger limit. What you really need to do use use the select.epoll object (still in the select module) instead of `select.