Question

Is it possible to use dispatchShell from a Controller?

My mission is to start a shell job when the user has signed up.

I’m using CakePHP 2.0

Answer 1

If you can’t mitigate the need to do this as dogmatic suggests then, read on.

So you have a (potentially) long-running job you want to perform and you don’t want the user to wait.

As the PHP code your user is executing happens during a request that has been started by Apache, any code that is executed will stall that request until it completion (unless you hit Apache’s request timeout).

If the above isn’t acceptable for your application then you will need to trigger PHP outwith the Apache request (ie. from the command line).

Usability-wise, at this point it would make sense to notify your user that you are processing data in the background. Anything from a message telling them they can check back later to a spinning progress bar that polls your application over ajax to detect job completion.

The simplest approach is to have a cronjob that executes a PHP script (ie. CakePHP shell) on some interval (at minimum, this is once per minute). Here you can perform such tasks in the background.

Some issues arise with background jobs however. How do you know when they failed? How do you know when you need to retry? What if it doesn’t complete within the cron interval.. will a race-condition occur?

The proper, but more complicated setup, would be to use a work/message queue system. They allow you to handle the above issues more gracefully, but generally require you to run a background daemon on a server to catch and handle any incoming jobs.

The way this works is, in your code (when a user registers) you insert a job into the queue. The queue daemon picks up the job instantly (it doesn’t run on an interval so it’s always waiting) and hands it to a worker process (a CakePHP shell for example). It’s instant and – if you tell it – it knows if it worked, it knows if it failed, it can retry if you want and it doesn’t accidentally handle the same job twice.

There are a number of these available, such as Beanstalkd, dropr, Gearman, RabbitMQ, etc. There are also a number of CakePHP plugins (of varying age) that can help:

• cakephp-queue (MySQL)
• CakePHP-Queue-Plugin (MySQL)
• CakeResque (Redis)
• cakephp-gearman (Gearman)
• and others.

I have had experience using CakePHP with both Beanstalkd (+ the PHP Pheanstalk library) and the CakePHP Queue plugin (first one above). I have to credit Beanstalkd (written in C) for being very lightweight, simple and fast. However, with regards to CakePHP development, I found the plugin faster to get up and running because:

• The plugin comes with all the PHP code you need to get started. With Beanstalkd, you need to write more code (such as a PHP daemon that polls the queue looking for jobs)
• The Beanstalkd server infrastructure becomes more complex. I had to install multiple instances of beanstalkd for dev/test/prod, and install supervisord to look after the processes).
• Developing/testing is a bit easier since it’s a self-contained CakePHP + MySQL solution. You simply need to type cake queue add user signup and cake queue runworker.