Question

In current lua sockets implementation, I see that we have to install a timer that calls back periodically so that we check in a non blocking API to see if we have received anything.

This is all good and well however in UDP case, if the sender has a lot of info being sent, do we risk loosing the data. Say another device sends a 2MB photo via UDP and we check socket receive every 100msec. At 2MBps, the underlying system must store 200Kbits before our call queries the underlying TCP stack.

Is there a way to get an event fired when we receive the data on the particular socket instead of the polling we have to do now?

Answer 1

There are a various ways of handling this issue; which one you will select depends on how much work you want to do.^*

But first, you should clarify (to yourself) whether you are dealing with UDP or TCP; there is no “underlying TCP stack” for UDP sockets. Also, UDP is the wrong protocol to use for sending whole data such as a text, or a photo; it is an unreliable protocol so you aren’t guaranteed to receive every packet, unless you’re using a managed socket library (such as ENet).

Lua51/LuaJIT + LuaSocket

Polling is the only method.

• Blocking: call socket.select with no time argument and wait for the socket to be readable.
• Non-blocking: call socket.select with a timeout argument of 0, and use sock:settimeout(0) on the socket you’re reading from.

Then simply call these repeatedly.
I would suggest using a coroutine scheduler for the non-blocking version, to allow other parts of the program to continue executing without causing too much delay.

Lua51/LuaJIT + LuaSocket + Lua Lanes (Recommended)

Same as the above method, but the socket exists in another lane (a lightweight Lua state in another thread) made using Lua Lanes (latest source). This allows you to instantly read the data from the socket and into a buffer. Then, you use a linda to send the data to the main thread for processing.

This is probably the best solution to your problem.

I’ve made a simple example of this, available here. It relies on Lua Lanes 3.4.0 (GitHub repo) and a patched LuaSocket 2.0.2 (source, patch, blog post re’ patch)

The results are promising, though you should definitely refactor my example code if you derive from it.

LuaJIT + OS-specific sockets

If you’re a little masochistic, you can try implementing a socket library from scratch. LuaJIT‘s FFI library makes this possible from pure Lua. Lua Lanes would be useful for this as well.

For Windows, I suggest taking a look at William Adam’s blog. He’s had some very interesting adventures with LuaJIT and Windows development. As for Linux and the rest, look at tutorials for C or the source of LuaSocket and translate them to LuaJIT FFI operations.

(LuaJIT supports callbacks if the API requires it; however, there is a signficant performance cost compared to polling from Lua to C.)

LuaJIT + ENet

ENet is a great library. It provides the perfect mix between TCP and UDP: reliable when desired, unreliable otherwise. It also abstracts operating system specific details, much like LuaSocket does. You can use the Lua API to bind it, or directly access it via LuaJIT’s FFI (recommended).

_{^* Pun unintentional.}