Question

I am using multiprocessing module in python2.7 and found to have some issue with it, which is currently going out of my head, as in how to resolve it.

The sample code that I am using is as follows:-

from multiprocessing import Pool

def func_eachcol(col_index):
    global list_of_lists

    for row in xrange(list_of_lists):
        list_of_lists[row][col_index] = 'Update Value'

class C1:
    def func_callmultiprocess(self):

        global list_of_lists
        col_len = len(cols)
        pool = Pool(col_len)
        for col in xrange(col_len):
            pool.apply_async(func_eachcol, args=(col))
        pool.close()
        pool.join()

        print list_of_lists

Basically, what i am doing, is on each col I am opening up a new process, and iterating through each row of list_of_lists, and then updating on that row,col in that very process.

The issue I am seeing here is that after multiprocessing is over, i cannot see the updated list_of_lists, infact it is the old list_of_lists.

but when i see, inside the multiprocess_function (func_eachcol), then I could see the values getting updated in that very index(row, col).

Please provide some remedy to this problem.

Added one more code, to simple what I wanted to achieve

from multiprocessing import Pool

def func_eachcol(col_index):
    print "Hello",
    global list

    print list,
    list[col_index] = -list[col_index]
    print list

class C1:
    def func_callmultiprocess(self):

        global list 
        list = [1,2,3]

        col_len = len(list)

        pool = Pool(col_len)
        for col in xrange(col_len):
            pool.apply_async(func_eachcol, args=(col,))

        pool.close()
        pool.join()

        print list

c = C1()
c.func_callmultiprocess()


**Ouput**
Hello [1, 2, 3] [-1, 2, 3]
Hello [1, 2, 3] [1, -2, 3]
Hello [1, 2, 3] [1, 2, -3]
[1, 2, 3]

Basically what i wanted the output in the end is [-1, -2, -3]. How is it possible.?

Answer 1

Global variables are not shared between multiple processes; you are manipulating a global list in a sub-process only, the parent process won’t see the changes.

Review the documentation on sharing state for options on how to pass information between processes. You probably want a Manager instance to share your list.