Question

I want to download a large file and concurrently handle other things.

However, luasocket.http never calls coroutine.yield(). Everything else freezes while the file downloads.

Here’s an illustrative example, in which I try to simultaneously download a file and print some numbers:

local http = require'socket.http'

local downloadRoutine = coroutine.create(function ()
    print 'Downloading large file'
    -- Download an example file
    local url = 'http://ipv4.download.thinkbroadband.com/5MB.zip'
    local result, status = http.request(url)
    print('FINISHED download ('..status..', '..#result..'bytes)')
end)

local printRoutine = coroutine.create(function ()
    -- Print some numbers
    for i=1,10 do
        print(i)
        coroutine.yield()
    end
    print 'FINISHED printing numbers'
end)

repeat
    local printActive = coroutine.resume(printRoutine)
    local downloadActive = coroutine.resume(downloadRoutine)
until not downloadActive and not printActive
print 'Both done!'

Running it produces this:

1
Downloading large file
FINISHED download (200, 5242880bytes)
2
3
4
5
6
7
8
9
10
FINISHED printing numbers
Both done!

As you can see, printRoutine is resumed first. It prints the number 1 and yields. The downloadRoutine is then resumed, which downloads the entire file, without yielding. Only then are the rest of the numbers printed.

I don’t want to write my own socket library! What can I do?

Edit (later the same day): Some MUSH users have also noticed. They provide helpful ideas.

Answer 1

I don’t see why you can’t use PiL advice or copas library (this is almost the same answer as is given here).

Copas wraps the socket interface (not socket.http), but you can use low level interface to get what you need with something like this (not tested):

require("socket")
local conn = socket.tcp()
conn:connect("ipv4.download.thinkbroadband.com", 80)
conn:send("GET /5MB.zip HTTP/1.1\n\n")
local file, err = conn:receive()
print(err or file)
conn:close()

You can then use addthread from copas to give you a non-blocking socket and use step/loop functions to do receive while there is something to receive.

Using copas is less work, while using settimeout(0) directly gives you more control.