Question

I want to make a HTTP request using Google App Engine in Go to a slow-responding website. My code is:

func request(w http.ResponseWriter, r *http.Request) {
    c:=appengine.NewContext(r)
    client := urlfetch.Client(c)
    resp, err := client.Get("http://www.example.com")
    if err != nil {
        log.Print("err %s", err.String())
        return
    }
    ...}

After calling my website, I get such an error:

API error 1 (urlfetch: INVALID_URL): ApplicationError: 5 timed out

After looking a bi into it, I found some reference to Transport class that allows to wait for HTTP requests for up to 60 seconds, but I can’t find any description or example on how to make that work.

How can I make a HTTP request in GAE Go that will have a deadline longer than 5 seconds?

Answer 1

I would recommend if possible, doing the urlfetch outside of your main request handler using Tasks Queue API.

When constructing urlfetch.Transport, you should be able to override the default deadline like this:

t := urlfetch.Transport{Context:c, DeadlineSeconds: 5.0}
client := http.Client{Transport: &t}