Question

I need to get a file into memory in my app from a secured web location. I have the URL of the file to capture, but can’t seem to get the security issue resolved. Here’s the code from the Cookbook samples page:

def download(address)
{
    def file = new FileOutputStream(address.tokenize("/")[-1])
    def out = new BufferedOutputStream(file)
    out << new URL(address).openStream()
    out.close()
}

and here’s my “memory” version of the same function which should return a byte array of the file’s contents:

def downloadIntoMem(address)
{  // btw, how frickin powerful is Groovy to do this in 3 lines (or less)
        def out = new ByteArrayOutputStream()
        out << new URL(address).openStream()
        out.toByteArray()
}

When I try this against an unsecured URL (pick any image file you can find on the net), it works just fine. However, if I pick a URL that requires a user/password, no go.

All right, done a bit more work on this. It seems that the Authenticator method does work, but in a round-about way. The first time I access the URL, I get a 302 response with a location to a login server. If I access that location with an Authenticator set, then I get another 302 with a Cookie and the location set back to the original URL. If I then access the original, the download occurs correctly.

So, I have to mimic a browser a bit, but eventually it all works.

Making this a community wiki, so others can add other methods.

Thanks!

Answer 1

If creds on url doesn’t work, you can use this. It works for Basic Authentication.

new File(localPath).withOutputStream { out ->
    def url = new URL(remoteUrl).openConnection()
    def remoteAuth = "Basic " + "${user}:${passwd}".bytes.encodeBase64()
    url.setRequestProperty("Authorization", remoteAuth);
    out << url.inputStream
}

Hope that helps!