Question

I ran into an error that was painful to track down, so I thought I’d add the cause + “solution” here.

The setup:
Devbox – Running Google App Engine listening on all ports (“–address=0.0.0.0”) serving a URL that launches a task.
Client – Client (Python requests library) which queries the callback URL

App Engine code:

class StartTaskCallback(webapp.RequestHandler):
  def post(self):
    param = self.request.get('param')
    logging.info('STARTTASK: %s' % param)
    # launch a task
    taskqueue.add(url='/tasks/mytask',
                  queue_name='myqueue',
                  params={'param': param})

class MyTask(webapp.RequestHandler):
  def post(self):
    param = self.request.get('param')
    logging.info('MYTASK: param = %s' % param)

When I queried the callback with my browser, everything worked, but the same query from the remote client gave me the following error:

ERROR    2012-03-23 21:18:27,351 taskqueue_stub.py:1858] An error occured while sending the task "task1" (Url: "/tasks/mytask") in queue "myqueue". Treating as a task error.
Traceback (most recent call last):
  File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/api/taskqueue/taskqueue_stub.py", line 1846, in ExecuteTask
    connection.endheaders()
  File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/httplib.py", line 868, in endheaders
    self._send_output()
  File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/httplib.py", line 740, in _send_output
    self.send(msg)
  File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/httplib.py", line 699, in send
    self.connect()
  File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/httplib.py", line 683, in connect
    self.timeout)
  File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/socket.py", line 498, in create_connection
    for res in getaddrinfo(host, port, 0, SOCK_STREAM):
gaierror: [Errno 8] nodename nor servname provided, or not known

This error would just spin in a loop as the task retried. Though oddly, I could go to Admin -> Task Queues and click ‘Run’ to get the task to complete successfully.

At first I thought this was an error with the binding. I would not get an error if I queried the StartTaskCallback via the browser or if I ran the client locally.

Finally I noticed that App Engine is using the ‘host’ field of the request in order to build an absolute URL for the task. In /Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/api/taskqueue/taskqueue_stub.py (1829):

  connection_host, = header_dict.get('host', [self._default_host])
  if connection_host is None:
    logging.error('Could not determine where to send the task "%s" '
                  '(Url: "%s") in queue "%s". Treating as an error.',
                  task.task_name(), task.url(), queue.queue_name)
    return False
  connection = httplib.HTTPConnection(connection_host)

In my case, I was using a special name + hosts file on the remote client to access the server.
192.168.1.208 devbox
So the ‘host’ for the remote client looked like ‘devbox:8085’ which the local server could not resolve.

Answer 1

To fix the issue, I simply added devbox to my AppEngine server’s hosts file, but it sure would have been nice if the gaierror exception had printed the name it failed to resolve, or if App Engine didn’t use the ‘host’ of the incoming request to build a URL for task creation.