Question

I have two sets of numbers, each in a list in my Python script. For each number in the first list, I need to see if any of the numbers in the second are larger than it. I only need the number of times that an n2 was larger than an n1. (For example, if numset1 is [7,2] and numset2 is [6,9], I just need 3)
Right now I’m doing this – going through each n1 and checking if each n2 is larger than it:

possibilities = [(n1<n2) for n1 in numset1 for n2 in numset2]
numPossibilities = sum(possibilities)

Currently this is the slowest portion of my script, particularly when dealing with larger datasets (numset1 and numset2 containing thousands of numbers). I’m sure there’s some way to make this more efficient, I’m just not sure how.

Answer 1

Sort numset2 and then iterate over numset1 but use a binary search on numset2, for example using the bisect module: http://docs.python.org/2/library/bisect.html

import bisect
# your code here
numset2.sort()
L = len(numset2)
numPossibilities = sum([bisect.bisect_right(numset2,n1) < L for n1 in numset1])

Also note that your original code does not compute what you have asked for in your second sentence – for each element in numset1, it sums how many elements in numset2 are greater than this element, not whether there is an element that matches the criterion.

To match your original code, do:

numPossibilities = sum([L - bisect.bisect_right(numset2,n1) for n1 in numset1])