Question

I’m putting together my first commercial PHP application, it’s nothing really huge as I’m still eagerly learning PHP 🙂

Right now I’m still in the conceptual stage of planning my application but I run into one problem all the time, the application is supposed to be self-hosted by my customers, on their own servers and will include some very long running scripts, depending on how much data every customer enters in his application.

Now I think I have two options, either use cronjobs, like for example let one or multiple cronjobs run at a time that every customer can set himself, OR make the whole processing of data as daemons that run in the background…

My question is, since it’s a self-hosted application (and every server is different)… is it even recommended to try to write php that starts background processes on a customers server, or is this more something that you can do reliably only on your own server…?

Or should I use cronjobs for these long running processes?

(depending on the amount of data my customers will enter in the application, a process could run 3+ hours)

Is that even a problem that can be solved, reliably, with PHP…? Excuse me if this should be a weird question, I’m really not experienced with PHP daemons and/or long running cronjobs created by php.

So to recap everything:
Commercial self-hosted application, including long running processes, cronjobs or daemons? And is either or maybe both also a reliable solution for a paid application that you can give to your customers with a clear conscience because you know it will work reliable on all kinds of different servers…?

EDIT*
PS: Sorry, I forgot to mention that the application targets only Linux servers, so everything like Debian, Ubuntu etc etc.

Answer 1

Short answer, no, don’t go for background process if this will be a client hosted solution. If you go towards the ASP concept (Application Service Provider… not Active Server Pages ;)) then you can do some wacky stuff with background processes and external apps connecting to your sql servers and processing stuff for you.

What i suggest is to create a strong task management backbone and link that to a solid task processing infrastructure. I’ll recommend you read an old post i did quite some time ago regarding background processes and a strategy i had adopted to fix long running processes:

Start & Stop PHP Script from Backend Administrative Webpage

Happy reading…

UPDATE

I realize that my old post is far from easy to understand so here goes:

You need 2 models: Job and JobQueue, 2 controller: JobProcessor, XYZProcessor

JobProcessor is called either by a user when a page triggers or using a cronjob as you wish. JobProcessor::process() is the key that starts the whole processing or continues it. It loads the JobQueues and asks the job queues if there is work to do. If there is work to do, it asks the jobqueue to start/continue it’s job.

JobQueue Model: Used to queue several JOBS one behind each other and controls what job is currently current by keep some kind of ID and STATE about which job is running.

Job Model: Represents exactly what needs to be done, it contains for example the name of the controller that will process the data, the function to call to process the data and a serialized configuration property that describe what must be done.

XYZController: Is the one that contains the processing method. When the processing method is called, the controller must load everything it needs to memory and then process each individual unit of work as fast as possible.

Example:

1. Call of index.php
2. Index.php creates a jobprocessor controller
3. Index.php calls the jobprocessor’s process()
4. JobProcessor::Process() loads all the queues and processes them
5. For each JobQueue::Process(), the job queue loads it’s possible Jobs and detects if one is currently running or not. If none is running, it starts the next one by calling Job::Process();
6. Job::Process() creates the XYZController that will work the task at hand. For example, my old system had an InvoicingController and a MassmailingController that worked hand in hand.
7. Job::Process() calls XYZController::Prepare() so that it loads it’s information to process. (For example, load a batch of emails to process, load a batch of invoices to create)
8. Job::Process() calls XYZController::RunWorkUnit() so that it processes a single unit of work (For example, create one invoice, send one email)
9. Job::Process() asks JobProcessingController::DoIStillHaveTimeToProcess() and if so, continues processing the next element.
10. Job::Process() runs out of time and calls XYZController::Cleanup() so that all resources are released
11. JobQueue::Process() ends and returns to JobController
12. JobController::Process() is about to end? Open a socket, call myself back so i can start another round of processing until i don’t have anything to do anymore
13. Handle the request from the user that start in position #1.

Ultimately, you can instead open a socket each time and ask the processor to do something, or you can queue a CronJob to call your processor. This way your users won’t get stuck waiting for the 3/4 work units to complete each time.