Question

We are currently using Google App Engine to evaluate solutions to Python problems submitted by students. We have moved all of the untrusted code execution off to a separate GAE application that doesn’t use the datastore. Everything seems to be working fine for the 50+ problems we have uploaded, but I’m curious what security holes remain that industrious students will find. How should we further protect this code from the untrusted code it execs in GAE?

#The solution and doctest are untrusted code passed in to the GAE app. 
solution = 'b=5'
doctest = '>>> b \n 5'

#Compile and exec the untrusted solution provided by the user. 
compiled = compile(solution, 'submitted code', 'exec')
sandbox = {}
exec compiled in sandbox

#Compile and exec each of the doctests
test_cases = doctest.DocTestParser().get_examples(doctest)
for test in test_cases:
  if not test.want:
    exec test.source in sandbox

Answer 1

Look at shell.appspot.com’s source — it actually even uses the datastore (for session persistence). At the core it’s basically just doing a simple exec just like you are — there are other refinements, but nothing special relative to “lockdown” of the untrusted code. Presumably the Google engineers (in the App Engine team) who released this sample code feel pretty good about its security against untrusted code.

The one feasible “attack” that I can think of is a “denial of service” attack that repeatedly sucks up resources and cause your account to be charged (if enabled for charging for beyond-the-free-quota usage) — as long as you’ve stored somewhere safe (in a different app) the identity of whoever’s uploaded a given piece of code, I don’t see why a student should try such a prank and risk expulsion or something.