I have about ten amazon ec2 servers all constantly writing to my mysql database from rake tasks.. and occasionally when I log into them to check their progress, I will find that they have stopped; not from any exception that has been raised, but leaving just the word “Killed” behind.
Even when i run the task with --trace, the only thing printed out is the word “Killed.”
I dont quite understand why this is happening (I have even set up rescue blocks to ensure that this is not an issue having to do with my database being locked, or any thing like that..)
I was hoping someone in the stackoverflow world would know how to either
1) Stop my rake tasks from being randomly ‘Killed’
or
2) Help me set up my servers to immediately relaunch the tasks if they are ‘Killed’
I suspect that they are victims of OOM killer. Maybe they take up too much memory and the system kills them in order to survive.
You could set up a god to watch over those rake tasks. If they get killed, god can relaunch them.