Question

This is surely no python-specific question, but I am looking for a python-specific answer – if any. It is about putting code blocks with a large number of variables into functions (or alike?). Let me assume this code

##!/usr/bin/env python
# many variables: built in types, custom made objects, you name it.
# Let n be a 'substantial' number, say 47.
x1 = v1
x2 = v2
...
xn = vn

# several layers of flow control, for brevity only 2 loops
for i1 in range(ri1):
    for i2 in range(ri2):
        y1 = f1(i1,i2)
        y2 = f2(i1,i2)
        # Now, several lines of work

        do_some_work

        # involving HEAVY usage and FREQUENT (say several 10**3 times)
        # access to all of x1,...xn, (and maybe y1,y2)
        # One of the main points is that slowing down access to x1,...,xn
        # will turn into a severe bottleneck for the performance of the code.


# now other things happen. These may or may not involve modification
# of x1,...xn

# some place later in the code, again, several layers of flow control,
# not necessarily identical to the first occur
for j1 in range(rj1):
    y1 = g1(j1)
    y2 = g2(j1)
    # Now, again

    do_some_work  # <---- this is EXACTLY THE SAME code block as above

# a.s.o.

Obviously I would like to put ‘do_some_work’ into something like a function (or maybe something better?).

What would be the most performant way to do this in python

• without function calls with a confusingly large numbers of arguments

• without performance lossy indirection to access x1,…,xn (Say, by wrapping them into another list, class, or alike)

• without using x1,…,xn as globals in a function do_some_work(…)

I have to admit, that I always find myself returning to globals.

Answer 1

A simple and dirty(probably not optimal) banchmark:

import timeit
def test_no_func():
    (x0,x1,x2,x3,x4,x5,x6,x7,x8,x9,x10,x11,x12,x13,x14,x15,x16,x17,x18,x19) = range(20)
    for i1 in xrange(100):
            for i2 in xrange(100):
                    for i3 in xrange(100):
                            results = [x0+x1+x2+x3+x4+x5+x6 for _ in xrange(100)]
                            results.extend(x7+x8+x9+x10+x11+x12+x13+x14+x15 for _ in xrange(100))
                            results.extend(x16+x17+x18+x19+x0 for _ in xrange(500))
    for j1 in xrange(100):
            for j2 in xrange(100):
                    for i3 in xrange(100):
                            results = [x0+x1+x2+x3+x4+x5+x6 for _ in xrange(100)]
                            results.extend(x7+x8+x9+x10+x11+x12+x13+x14+x15 for _ in xrange(100))
                            results.extend(x16+x17+x18+x19+x0 for _ in xrange(500))


def your_func(x_vars):
    # of the number is not too big you can simply unpack.
    # 150 is a bit too much for unpacking...
    (x0,x1,x2,x3,x4,x5,x6,x7,x8,x9,x10,x11,x12,x13,x14,x15,x16,x17,x18,x19) = x_vars

    results = [x0+x1+x2+x3+x4+x5+x6 for _ in xrange(100)]
    results.extend(x7+x8+x9+x10+x11+x12+x13+x14+x15 for _ in xrange(100))
    results.extend(x16+x17+x18+x19+x0 for _ in xrange(500))
    return results


def test_func():
    (x0,x1,x2,x3,x4,x5,x6,x7,x8,x9,x10,x11,x12,x13,x14,x15,x16,x17,x18,x19) = range(20)
    for i1 in xrange(100):
            for i2 in xrange(100):
                    for i3 in xrange(100):
                            results = your_func(val for key,val in locals().copy().iteritems() if key.startswith('x'))
    for j1 in xrange(100):
            for j2 in xrange(100):
                    for i3 in xrange(100):
                            results = your_func(val for key,val in locals().copy().iteritems() if key.startswith('x'))


print timeit.timeit('test_no_func()', 'from __main__ import test_no_func', number=1)
print timeit.timeit('test_func()', 'from __main__ import test_func, your_func', number=1)

Result:

214.810357094
227.490054131

which is about 5% slower passing the arguments. But probably you can’t do much better than this introducing 1 million function calls…