Question

The Problem

I have a CakePHP application in which I call a stored procedure:

$this->FileUpload->query('exec sproc_runjob_ProcessTests', false);

This procedure picks up an uploaded file and processes it. Sometimes people upload multiple files, which calls this once for each file uploaded.

The problem is that I’m getting this error:

Database Error
Error: SQLSTATE[42000]:
    [Microsoft][SQL Server Native Client 11.0][SQL Server]
    SQLServerAgent Error: Request to run job ProcessTests
    (from User [redacted]) refused because the job already has
    a pending request from User [redacted].
SQL Query: exec sproc_runjob_ProcessTests

As far as I can tell, this means that the stored procedure can’t run concurrently – I would have to run the sproc for each process serially, which would be a shame on our fancy new multi-core server.

Solutions we’ve thought of

1. Run the sproc serially

   Basically the idea here is to call the stored procedure once, then ignore it’s failure if it’s already running. At the end of the sproc call, it would check the database to see if it needs to run again (if there are any "new" files) and run itself again if that is the case.

2. Don’t use the existing stored procedure

   This would be difficult because of time restrictions, but if someone has a good suggestion for the way we should be doing this, I’d be happy to hear it.

Thanks in advance!

Search results

1. Looks like this guy has the same problem, and solves it by checking the job status before calling. Perhaps I could do this from PHP before I make the sproc call?

2. Here’s a mention of something called Service Broker. This might actually be what I’m looking for, but I’d still love any input.

Thanks in advance!

Answer 1

From the error message your getting, sounds like your stored procedure is trying to start a SQL job that’s registered in SQL agent. If that’s true, then I would recommend another strategy.

Ideally, you should pass the file directly to your stored procedure as a binary type, such as image or blob. Let the multithreading of the database handle scaling of concurrent requests. Otherwise, if you try to process the files serially, the application won’t scale well.

Service Broker is really an endpoint that allows your app to connect to the database in a different way. Yes, you can use it – it would work, but I causation against it.

Alternatively, you can continue to store files on the filesystem, and the only store the path to your file in the database.

Good luck!