Question

I have an 2 dimensional array. Each of the row vectors, in this case, is considered a quantity of interest. What I want to do is return all the rows that appear exactly once as one array, and all the rows that appear more than once as a second array.

For example, if the array was:

a=[[1,1,1,0], [1,1,1,0], [5,1,6,0], [3,2,1,0], [4,4,1,0], [5,1,6,0]]

I would like to return two arrays:

nonsingles=[[1,1,1,0], [1,1,1,0], [5,1,6,0], [5,1,6,0]]
singles= [[3,2,1,0], [4,4,1,0]]

It is important that the order stay preserved. The code I have written to do this is as follows:

def singles_nonsingles(array):
#returns the elements that occur only once, and the elements
#that occur more than once in the array
singles=[]
nonsingles=[]
arrayhash=map(tuple, array)

for x in arrayhash:
    if (arrayhash.count(x)==1):
        singles.append(x)

    if (arrayhash.count(x)>1):
        nonsingles.append(x)

nonsingles=array(nonsingles)
singles=array(singles)

return {'singles':singles, 'nonsingles':nonsingles}

Now, I am happy to say that this works, but unhappy to say that it is extremely slow, as a typical array i have is 30000(rows)x10 elements/row=300000 elements. Can anyone give me some tips about how to speed this up?? I apologize if this question is very simple, I am new to Python. Also, I am using Numpy/Scipy with Python 2.7, if that is any help.

Answer 1

In Python 2.7 or above, you can use collections.Counter to count the number of occurrences:

def unique_items(iterable):
    tuples = map(tuple, iterable)
    counts = collections.Counter(tuples)
    unique = []
    non_unique = []
    for t in tuples:
        if counts[t] == 1:
            unique.append(t)
        else:
            non_unique.append(t)
    return unique, non_unique