Question

I am using the accepted answer from this question. The relevant code is below.

import multiprocessing

def query_with_timeout(dbc, timeout, query, *a, **k):
  conn1, conn2 = multiprocessing.Pipe(False)
  subproc = multiprocessing.Process(target=do_query,
                                    args=(dbc, query, conn2)+a, 
                                    kwargs=k)
  subproc.join(timeout)
  if conn1.poll():
    return conn1.recv()
  subproc.terminate()
  raise Exception("Query %r ran for >%r" % (query, timeout))

def do_query(dbc, query, conn, *a, **k):
  cu = dbc.cursor()
  cu.execute(query, *a, **k)
  return cu.fetchall()

My call looks like this:

res = query_with_timeout(dbconn, 30, "SELECT * FROM `table`)
print res

dbconn is a connection object. It is used elsewhere in the application (outside of this timeout function) without any problem.

It appears that it joins the subprocess and waits for 30 seconds. It never enters the if conn1.poll():
return conn1.recv() block of code though when it polls for results. Instead, I always receive the exception.

I know that the query runs (it’s a simple select), and it runs in less than a second.

Did I miss something?

Answer 1

Well, a couple of things based on the code you’ve posted. First, you don’t ever start() the process. But perhaps that’s just a mistake you made in typing up this example?

Second, you don’t ever send() anything via the connection.

Here’s a simplified version of the above with expected behavior:

import multiprocessing
import time

def simple_example(wait, timeout, query):
    conn1, conn2 = multiprocessing.Pipe(False)
    subproc = multiprocessing.Process(target=do_query, args=(wait, query, conn2))
    subproc.start()
    subproc.join(timeout)
    if conn1.poll():
        return conn1.recv()
    subproc.terminate()
    raise Exception('Query %r ran for >%r' % (query, timeout))

def do_query(wait, query, conn):
    print query
    time.sleep(wait)
    conn.send(query)

res = simple_example(0, 2, 'foo')
res = simple_example(3, 2, 'foo')

I fear this may simply be a problem with your example code though.