Question

I’m implementing an offline task queue in PHP. Online pages insert a record into this table to schedule work:

create table sl_task_queue (
    task_id int primary key auto_increment,
    task_data varchar(255) not null,
    started_date datetime,
    completed_date datetime,
    priority int not null default 10
) ENGINE=Innodb;

Then, offline, I have a pool of PHP CLI processes which sit in a loop waiting for work to do. To query for work, they run this function:

$SEMKey = "849039";
$seg = sem_get( $SEMKey, 2, 0666, -1) ; 

function getTask() {
    sem_acquire($GLOBALS['seg']);

    // This function returns null for no results, or a Task ORM object on success
    $Task = eSqlObject("Task", "select * from sl_task_queue 
        where started_date is null 
        order by task_id asc limit 1 for update"); 

    if( $Task ) {
        // Set started_date to claim this task
        $Task = new LucidityTask($hash);
        $Task['started_date'] = date("Ymd H:i:s");
        $Task->save(); // performs an UPDATE query
    }

    sem_release($GLOBALS['seg']);   

    return $Task;
}

I’ve tried two things here, a PHP semaphore and MySQL’s “SELECT … FOR UPDATE” feature. I’ve also tried a full LOCK TABLE command. Despite this, I still have problems with two processes in the daemon claiming the same row, and I can’t pin down why. A couple suspicions – do PHP’s semaphores go into derp mode when you do separate CLI processes? Or is MySQL taking its sweet time to actually update the underlying table data, so that after this function exits, a SELECT query would still return the old data for a short time?

Any other ideas?

Thanks so much!

Answer 1

GAH! This is why copy/paste is evil. Here’s the problem:

$seg = sem_get( $SEMKey, 2, 0666, -1) ; 

The “2” allows two processes to acquire the semaphore at once. Thanks, Louis H.!