Question

I have a heavy processing that needs to be finished using a lot of machines. Currently, Resque have 700 workers running, and everything is working. However if I increase the number of workers to 1000, I start getting a lot of redis timeouts.
I am almost sure that the limitation is in redis, because I cannot even connect from redis-cli anymore.
Is there any limit either in resque or redis? Or it could be some filesystem limit?

Does anyone know where would it be this configuration so i could change it?

thanks.

Answer 1

You’re probably hitting resources limit on your server box.

On Linux, check :

ulimit -a
sysctl net.ipv4.ip_local_port_range
sysctl net.ipv4.tcp_fin_timeout
sysctl net.core.somaxconn
sysctl net.ipv4.tcp_tw_recycle
sysctl net.ipv4.tcp_tw_reuse
sysctl fs.file-max
sysctl net.ipv4.tcp_window_scaling
sysctl kernel.pid_max
sysctl net.ipv4.tcp_orphan_retries
sysctl net.ipv4.tcp_max_orphans

On FreeBSD, check :

sysctl kern.ipc.nmbclusters
sysctl kern.ipc.maxsockets
sysctl kern.maxfiles
sysctl kern.maxfilesperproc
sysctl net.inet.tcp.msl
sysctl net.inet.ip.portrange.first
sysctl net.inet.ip.portrange.last

Get more information about those sysctl configurations and increase as needed.