Question

Given two sorted arrays like the following:

a = array([1,2,4,5,6,8,9])

b = array([3,4,7,10])

I would like the output to be:

c = array([1,2,3,4,5,6,7,8,9,10])

or:

c = array([1,2,3,4,4,5,6,7,8,9,10])

I’m aware that I can do the following:

c = unique(concatenate((a,b))

I’m just wondering if there is a faster way to do it as the arrays I’m dealing with have millions of elements.

Any idea is welcomed. Thanks

Answer 1

Since you use numpy, I doubt that bisec helps you at all… So instead I would suggest two smaller things:

1. Do not use np.sort, use c.sort() method instead which sorts the array in place and avoids the copy.
2. np.unique must use np.sort which is not in place. So instead of using np.unique do the logic by hand. IE. first sort (in-place) then do the np.unique method by hand (check also its python code), with flag = np.concatenate(([True], ar[1:] != ar[:-1])) with which unique = ar[flag] (with ar being sorted). To be a bit better, you should probably make the flag operation in place itself, ie. flag = np.ones(len(ar), dtype=bool) and then np.not_equal(ar[1:], ar[:-1], out=flag[1:]) which avoids basically one full copy of flag.
3. I am not sure about this. But .sort has 3 different algorithms, since your arrays maybe are almost sorted already, changing the sorting method might make a speed difference.

This would make the full thing close to what you got (without doing a unique beforehand):

def insort(a, b, kind='mergesort'):
    # took mergesort as it seemed a tiny bit faster for my sorted large array try.
    c = np.concatenate((a, b)) # we still need to do this unfortunatly.
    c.sort(kind=kind)
    flag = np.ones(len(c), dtype=bool)
    np.not_equal(c[1:], c[:-1], out=flag[1:])
    return c[flag]