Question

I have a Python function that adds a task queue for each email address in my mailing list (thousands each time I mail them). The problem is that, even though each mail was to be sent via execution in a task queue, I still get this dreaded error:

DeadlineExceededError: The API call mail.Send() took too long to respond and was cancelled.

Any solutions?

Answer 1

The deadline happens during communication between your GAE instance and the RPC server that handles the mail.Send call. This, in turn, might indicate internal problems of GAE or (more likely) failure to communicate with SMTP server in timely manner.

The latter is conceptually very similar to deadline on URLFetch call. It is possible, however, to set a custom deadline for URLFetch which largely alleviates that problem.

Unfortunately, there is no documented analogy for Mail API. There is workaround, though, that involves providing your own make_sync_call method – which allow for more lenient deadline – as parameter of EmailMessage.send(). To produce such a method, you need to delve into the internals of Python’s interface used to make GAE RPC calls. The solution I find working looks as follows:

from google.appengine.api import apiproxy_stub_map
from google.appengine.api.apiproxy_stub_map import APIProxyStubMap

class TimeoutAPIStub( object ):
    def __init__( self, service, deadline = 25 ):
        self.service = service
        self.deadline = deadline

    def CreateRPC( self ):
        rpc = apiproxy_stub_map.CreateRPC( self.service )
        rpc.deadline = self.deadline
        return rpc

def create_MakeSyncCall( service, deadline = 25 ):
    apsm = APIProxyStubMap()
    apsm.RegisterStub( service, TimeoutAPIStub( service, deadline ) )
    return apsm.MakeSyncCall

You can then use it to supply your custom deadline which will be honored by the resulting make_sync_call method:

msg = EmailMessage()
# ... prepare your email ...
msg.send(make_sync_call = create_MakeSyncCall('mail', deadline = 300))

If you want to know more about what happens under the curtains of GAE RPC calls, I suggest reading Nick Johnson’s blog post about it. This is good starting point if you’d ever want to go through the Python’s GAE RPC bindings in order to solve similar issues.