Question

I can run tasks as part of our functional tests just fine using this code.

def run_tasks():
    taskq = apiproxy_stub_map.apiproxy.GetStub('taskqueue')
    tasks = taskq.GetTasks("default")
    taskq.FlushQueue("default")
    try:
        while tasks:
            for task in tasks:
                url, body, headers, method = task["url"], \
                                             base64.b64decode(task["body"]), \
                                             task["headers"], \
                                             task["method"]
                try:
                    res = Request.blank(url,
                                        body=body,
                                        headers=headers,
                                        method=method).get_response(wsgi_app)
                    if res.status_int != 200:
                        log.error("%s\n%s" % (url, str(res)))
                except Exception:
                    log.error(url, exc_info=True)

            tasks = taskq.GetTasks("default")
            taskq.FlushQueue("default")
    finally:
        pass

However, pipeline tasks bomb out here with a message of

“Received evaluation task for pipeline ID “[some id]” on attempt 1, ‘
‘which will not be ready until: [sometime later in the future]”

The relevant source is here.

+++ b/src/pipeline/pipeline/pipeline.py Wed Apr 04 20:01:12 2012 -0400
@@ -1946,6 +1952,7 @@
     if pipeline_record.next_retry_time is not None:
       retry_time = pipeline_record.next_retry_time - _RETRY_WIGGLE_TIMEDELTA
       if self._gettime() <= retry_time:
+        # error under unit tests
         detail_message = (
             'Received evaluation task for pipeline ID "%s" on attempt %d, '
             'which will not be ready until: %s' % (pipeline_key.name(),

I thought of monkey patching _RETRY_WIGGLE_TIMEDELTA in the code that
brings up the appengine apis in my testsuite but that doesn’t seem to
be enough.

Short of waiting until it’s time, in my code which would make my
test suite run prohibitively slow, I’m at a loss.

Any ideas?

Answer 1

Well I mis-diagnosed the problem. A task was failing and that logic is the re-try logic. It’s kind of odd that that’s enforced in the pipeline api, but I’m sure there’s a good reason for it. So Somewhere in my loop of processing tasks I need to check for re-try count and somehow get those out of the queue, then raise all the errors information captured from the first runs so that the test can be considered failed.