We are using the Love2d Lua game engine that exposes a graphics api to Lua. We are trying to serialize a giant hash table that contains all the save game data for the game world. This hash includes some functions, and some of these functions call Love2d C functions.
In order to serialize the functions in the hash, we use string.dump, and load them back in with loadstring. This works well for pure Lua functions, but when we try to serialize and then load back in a function which calls a wrapped C function such as one in the Love2d api, loadstring returns nil.
Consider the following simple program that draws “hello, world” to the screen via Love2d’s graphics engine:
function love.load()
draw = function()
love.graphics.print('hello, world', 10, 10)
end
end
function love.draw()
draw()
end
We would like to be able to do this:
function love.load()
draw_before_serialize = function()
love.graphics.print('hello, world', 10, 10)
end
out = io.open("serialized.lua", "wb")
out:write('draw = load([[' .. string.dump(draw_before_serialize) .. ']])')
out:close()
require "serialized"
end
function love.draw()
draw()
end
Doing this writes to a Lua file on disk that contains a mix of non-compiled Lua and Lua bytecode, which looks something like this:
draw = load([[^[LJ^A^@
@main.lua2^@^@^B^@^B^@^D^E^B^B4^@^@^@%^A^A^@>^@^B^AG^@^A^@^Qhello, world
print^A^A^A^B^@^@]])
This method works fine with Lua functions that do not call C modules. We think that this is the problem because this example does work:
function love.load()
draw_before_serialize = function()
print('hello, world')
end
out = io.open("serialized.lua", "wb")
out:write('draw = load([[' .. string.dump(draw_before_serialize) .. ']])')
out:close()
require "serialized"
end
function love.draw()
draw()
end
Instead of calling the Love2d graphics method, it does a print to the console.
After more testing, we were confused to find that this example does work:
function love.load()
draw_before_serialize = function()
love.graphics.print('hello, world', 10, 10)
end
draw = load(string.dump(draw_before_serialize))
end
function love.draw()
draw()
end
Here we don’t actually write out the function to disk, and instead just dump it and then immediately load it back. We thought that perhaps the culprit was not writing out the data with the binary write mode flag set ("wb"), but since we are on Linux this flag has no effect.
Any ideas?
I think the problem is in the formatting of the string. Nicol Bolas might be right about the [[]] quote marks surrounding your byte-code dump, but this points at a bigger problem; The byte code really could be anything, but you’re treating it like it’s a normal string that can be written to and read from a text file. This problem is demonstrated by your last demo, where you load the dumped string without ever writing it to file.
This implementation of a serializer for tables which include functions kind of does what you want, I think, but I also think it’s broken (well, I couldn’t get it to work right anyway…). Anyway it’s on the right track. You need to format the bytecode and then write it to the file.
I’m sure there’s a better way to do it, but this works:
This loops through each character in the bytecode, formats them as escaped bytes (each is a string containing a code like “\097”, which upon interpolation would escape to “a”).
Line 4 of this sample is kind of dense so I’ll break it down. First,
pulls the i’th character out of the string. Then
gives back the ascii integer representation of the i’th character. Then we format it with
which gives us a string like “\ 97”, for example, if the character were “a”. But this won’t escape properly because we need “\097″, so we do a gsub replacing ” ” with “0”. The gsub returns the resulting string and the number of substitutions that were performed, so we just take the first return value and put it in “dec”. I’m not sure why the “%3d” format doesn’t replace the spaces with “0”‘s by default… oh well.
Then in order to execute the formatted binary string, we need to escape it and pass the result to “load”. The weirdo [[]] quote marks in Lua don’t do escapes like “”… in fact I’m not sure they do any escapes at all. So then to make an executable Lua string that will return a function that will do whatever is in “some_function”, we do this:
Ok – so putting all that together, I think we can make your test-case work like so:
When I run this with Love I get an OpenGL screen with “hello world” printed in the corner. The resulting file “serialized.lua” contains the following: