Question

How can I recreate the functionality of setfenv in Lua 5.2? I’m having some trouble understanding exactly how you are supposed to use the new _ENV environment variable.

In Lua 5.1 you can use setfenv to sandbox any function quite easily.

--# Lua 5.1

print('_G', _G)             -- address of _G

local foo = function()  
    print('env', _G)        -- address of sandbox _G
    bar = 1
end

-- create a simple sandbox
local env = { print = print }
env._G = env

-- set the environment and call the function
setfenv(foo, env)
foo()

-- we should have global in our environment table but not in _G
print(bar, env.bar)

Running this example shows an output:

_G    table: 0x62d6b0
env   table: 0x635d00
nil   1

I would like to recreate this simple example in Lua 5.2. Below is my attempt, but it does not work like the above example.

--# Lua 5.2

local function setfenv(f, env)
    local _ENV = env or {}       -- create the _ENV upvalue
    return function(...)
        print('upvalue', _ENV)   -- address of _ENV upvalue
        return f(...)
    end
end

local foo = function()
    print('_ENV', _ENV)          -- address of function _ENV
    bar = 1
end

-- create a simple sandbox
local env = { print = print }
env._G = env

-- set the environment and call the function
foo_env = setfenv(foo, env)
foo_env()

-- we should have global in our envoirnment table but not in _G
print(bar, env.bar)

Running this example shows the output:

upvalue    table: 0x637e90
_ENV       table: 0x6305f0
1          nil

I am aware of several other questions on this subject, but they mostly seem to be dealing with loading dynamic code (files or string) which work quite well using the new load function provided in Lua 5.2. Here I am specifically asking for a solution to run arbitrary functions in a sandbox. I would like to do this without using the debug library. According to the Lua documentation we should not have to rely on it.

Answer 1

You cannot change the environment of a function without using the debug library from Lua in Lua 5.2. Once a function has been created, that is the environment it has. The only way to modify this environment is by modifying its first upvalue, which requires the debug library.

The general idea with environments in Lua 5.2 is that the environment should be considered immutable outside of trickery (ie: the debug library). You create a function in an environment; once created there, that’s the environment it has. Forever.

This is how environments were often used in Lua 5.1, but it was easy and sanctioned to modify the environment of anything with a casual function call. And if your Lua interpreter removed setfenv (to prevent users from breaking the sandbox), then the user code can’t set the environment for their own functions internally. So the outside world gets a sandbox, but the inside world can’t have a sandbox within the sandbox.

The Lua 5.2 mechanism makes it harder to modify the environment post function-creation, but it does allow you to set the environment during creation. Which lets you sandbox inside the sandbox.

So what you really want is to just rearrange your code like this:

local foo;

do
  local _ENV = { print = print }

  function foo()
    print('env', _ENV)
    bar = 1
  end
end

foo is now sandboxed. And now, it’s much harder for someone to break the sandbox.

As you can imagine, this has caused some contention among Lua developers.