Question

I hope I’m able to explain this problem clearly. I’m a python experimenter (just in case the below query appears naive)

Assume that I have a dataset of the form:

a = ( ('309','308','308'), ('309','308','307'), ('308', '309','306', '304'))

Let me call each ('309','308','308') as a path.

I want to find the count of:

a. Count('309','308', <any word>)

b. Count('309',<any word>,'308')

and all possible permutations.

I’m thinking its some kind of a regex which will help me achieve this search. And, the number of paths I have goes onto 50000.

Can anyone suggest how I can do this kind of an operation in python? I explored trie, radix but I dont think that’ll help me.

Thanks,
Sagar

Answer 1

You could use collections.Counter to do this:

>>> from collections import Counter
>>> a = ( ('309','308','308'), ('309','308','307'), ('308', '309','306', '304'))
>>> Counter((x, y) for (x, y, *z) in a)
Counter({('309', '308'): 2, ('308', '309'): 1})
>>> Counter((x, z) for (x, y, z, *w) in a)
Counter({('308', '306'): 1, ('309', '308'): 1, ('309', '307'): 1})

I’m also using extended tuple unpacking here, which didn’t exist pre-Python 3.x, which is only needed if you have tuples of an uncertain length. In python 2.x, you could instead do:

Counter((item[0], item[1]) for item in a)

I couldn’t say how efficient this would be, however. I don’t believe it should be bad.

A Counter has a dict-like syntax:

>>> count = Counter((x, y) for (x, y, *z) in a)
>>> count['309', '308']
2

Edit: You mentioned they might be of any length greater than one, in this case, you could run into problems as they won’t be able to unpack if they are shorter than the required length. The solution is to change the generator expression to ignore any not in the required format:

Counter((item[0], item[1]) for item in a if len(item) >= 2)

E.g:

>>> a = ( ('309',), ('309','308','308'), ('309','308','307'), ('308', '309','306', '304'))
>>> Counter((x, y) for (x, y, *z) in a)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.2/collections.py", line 460, in __init__
    self.update(iterable, **kwds)
  File "/usr/lib/python3.2/collections.py", line 540, in update
    _count_elements(self, iterable)
  File "<stdin>", line 1, in <genexpr>
ValueError: need more than 1 value to unpack
>>> Counter((item[0], item[1]) for item in a)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.2/collections.py", line 460, in __init__
    self.update(iterable, **kwds)
  File "/usr/lib/python3.2/collections.py", line 540, in update
    _count_elements(self, iterable)
  File "<stdin>", line 1, in <genexpr>
IndexError: tuple index out of range
>>> Counter((item[0], item[1]) for item in a if len(item) >= 2)
Counter({('309', '308'): 2, ('308', '309'): 1})

If you need to have a variable length count, the easiest way is to use a list slice:

start = 0
end = 2
Counter(item[start:end] for item in a if len(item) >= start+end)

Of course, this only works for continuous runs, if you want to pick columns individually, you have to do a little more work:

def pick(seq, indices):
    return tuple([seq[i] for i in indices])

columns = [1, 3]
maximum = max(columns)
Counter(pick(item, columns) for item in a if len(item) > maximum)