Question

In designing my GWT/GAE app, it has become evident to me that my client-side (GWT) will be generating three types of requests:

• Synchronous – “answer me right now! I’m important and require a real-time response!!!”
• Asynchronous – “answer me when you can; I need to know the answer at some point but it’s really not all that ugent.”
• Command – “I don’t need an answer. This isn’t really a request, it’s just a command to do something or process something on the server-side.”

My game plan is to implement my GWT code so that I can specify, for each specific server-side request (note: I’ve decided to go with RequestFactory over traditional GWT-RPC for reasons outside the scope of this question), which type of request it is:

• SynchronousRequest – Synchronous (from above); sends a command and eagerly awaits a response that it then uses to update the client’s state somehow
• AsynchronousRequest – Asynchronous (from above); makes an initial request and somehow – either through polling or the GAE Channel API, is notified when the response is finally received
• CommandRequest – Command (from above); makes a server-side request and does not wait for a response (even if the server fails to, or refuses to, oblige the command)

I guess my intention with SynchronousRequest is not to produce a totally blocking request, however it may block the user’s ability to interact with a specific Widget or portion of the screen.

The added kicker here is this: GAE strongly enforces a timeout on all of its frontend instances (60 seconds). Backend instances have much more relaxed constraints for timeouts, threading, etc. So it is obvious to me that AsynchronousRequests and CommandRequests should be routed to backend instances so that GAE timeouts do not become an issue with them.

However, if GAE is behaving badly, or if we’re hitting peak traffic, or if my code just plain sucks, I have to account for the scenario where a SynchronousRequest is made (which would have to go through a timeout-regulated frontend instance) and will timeout unless my GAE server code does something fancy. I know there is a method in the GAE API that I can call to see how many milliseconds a request has before its about to timeout; but although the name of it escapes me right now, it’s what this “fancy” code would be based off of. Let’s call it public static long GAE.timeLeftOnRequestInMillis() for the sake of this question.

In this scenario, I’d like to detect that a SynchronousRequest is about to timeout, and somehow dynamically convert it into an AsynchronousRequest so that it doesn’t time out. Perhaps this means sending an AboutToTimeoutResponse back to the client, and force the client to decide about whether to resend as an AsynchronousRequest or just fail. Or perhaps we can just transform the SynchronousRequest into an AsynchronousRequest and push it to a queue where a backend instance will consume it, process it and return a response. I don’t have any preferences when it comes to implementation, so long as the request doesn’t fail or timeout because the server couldn’t handle it fast enough (because of GAE-imposed regulations).

So then, here is what I’m actually asking here:

• How can I wrap a RequestFactory call inside SynchronousRequest, AsynchronousRequest and CommandRequest in such a way that the RequestFactory call behaves the way each of them is intended? In other words, so that the call either partially-blocks (synchronous), can be notified/updated at some point down the road (asynchronous), or can just fire-and-forget (command)?
• How can I implement my requirement to let a SynchronousRequest bypass GAE’s 60-second timeout and still get processed without failing?

Please note: timeout issues are easily circumvented by re-routing things to backend instances, but backends don’t/can’t scale. I need scalability here as well (that’s primarily why I’m on GAE in the first place!) – so I need a solution that deals with scalable frontend instances and their timeouts. Thanks in advance!

Answer 1

If the computation that you want GAE to do is going to take longer than 60 seconds, then don’t wait for the results to be computed before sending a response. According to your problem definition, there is no way to get around this. Instead, clients should submit work orders, and wait for a notification from the server when the results are ready. Requests would consist of work orders, which might look something like this:

class ComputeDigitsOfPiWorkOrder {
  // parameters for the computation
  int numberOfDigitsToCompute;

  // Used by the GAE app to contact the requester when results are ready.
  ClientId clientId;
}

This way, your GAE app can respond as soon as the work order is saved (e.g. in Task Queue), and doesn’t have to wait until it actually finishes calculating a billion digits of pi before responding. Your GWT client then waits for the result using the Channel API.

In order to give some work orders higher priority, you can use multiple task queues. If you want Task Queue work to scale automatically, you’ll want to use push queues. Implementing priority using push queues is a little tricky, but you can configure high priority queues to have faster feed rate.

You could replace Channel API with some other notification solution, but that would probably be the most straightforward.