Question

I have a wrapper python script which repeatedly calls another python script via os.system. This works well enough, but there is quite a performance hit invoking the sub-shell and importing the modules again and again. How might I transform this to something more elegant and performant?

counter = 0
for thing in list_of_stuff:
    os.system("python inner_script.py %s result_%s" % (thing, counter)
    counter += 1

I would prefer to do this all in the wrapper, but can modify inner-script.py if that’s the only or best way.

If it’s relevant, the environment is Python 2.7 on Windows.

edit: I don’t just import inner_script because it doesn’t understand the command line parameters:

import inner_script    
counter = 0
for thing in ['TR2','TR5']:
    inner_script('%s result_%s' % (thing, counter))
counter += 1

result:

C:\> python xx-wrapper.py

inner_script [input features] [output workspace]

which is the usage message returned by inner_script.py:

if len(sys.argv) < 3:
    print usage
    exit()

in_features = sys.argv[1]
out_folder  = sys.argv[2]

main(in_features, out_folder)

Answer 1

In inner-script.py create a class around the task that the script does.

for thing in list_of_stuff:
    x = __import__('inner-script')
    x.className(thing, counter)

This should be efficient enough.

---

Or, if you can’t/wont modify the inner-script.py, here’s a way to call it from within python and pass variables as local() or global() variables which the script can read from.. just as if you passed the params upon execution in your example:

x = __import__('inner-script', globals(), (thing, counter), [], -1)

Then whatever stuff you want to be, more info can be found here