Question

I have simple map-reduce type algorithm, which I want to implement in python and make use of multiple cores.

I read somewhere that threads using native thread module in 2.6 dont make use of multiple cores. is that true?

I even implemented it using stackless python however i am getting into weird errors
[Update: a quick search showed that the stack less does not allows multiple cores
So are their any other alternatives?]

def Propagate(start,end):
print "running Thread with range: ",start,end
def maxVote(nLabels):
    count = {}
    maxList = []
    maxCount = 0
    for nLabel in nLabels:
        if nLabel in count:
            count[nLabel] += 1
        else:
            count[nLabel] = 1
    #Check if the count is max
        if count[nLabel] > maxCount:
            maxCount = count[nLabel];
            maxList = [nLabel,]
        elif count[nLabel]==maxCount:
            maxList.append(nLabel)
    return random.choice(maxList)        

for num in range(start,end):
    node=MapList[num]
    nLabels = [Label[k] for k in Adj[node]]
    if (nLabels!=[]):
        Label[node] = maxVote(nLabels)
    else:
        Label[node]=node

However in above code the values assigned to Label, that is the change in dictionary are lost.

Above propagate function is used as callable for MicroThreads (i.e. TaskLets)

Answer 1

Use the multiprocessing module of the standard library — it mimics the interface of the threading module (to ease porting of existing multithreaded code) and can fully use all the cores you have. threading and stackless are both single-core (and indeed, stackless is non-preemptive, so programming for it is quite different than programming for either threading or multiprocessing).