Question

I’m rather new to NumPy. Anyone have an idea for making this code, especially the nested loops, more compact/efficient? BTW, dist and data are three-dimensional numpy arrays.

def interpolate_to_distance(self,distance):

    interpolated_data=np.ndarray(self.dist.shape[1:])
    for j in range(interpolated_data.shape[1]):
        for i in range(interpolated_data.shape[0]):
            interpolated_data[i,j]=np.interp(
                                  distance,self.dist[:,i,j],self.data[:,i,j])

    return(interpolated_data)

Thanks!

Answer 1

Alright, I’ll take a swag with this:

def interpolate_to_distance(self, distance):
    dshape = self.dist.shape
    dist = self.dist.T.reshape(-1, dshape[-1])
    data = self.data.T.reshape(-1, dshape[-1])
    intdata = np.array([np.interp(distance, di, da)
                        for di, da in zip(dist, data)])
    return intdata.reshape(dshape[0:2]).T

It at least removes one loop (and those nested indices), but it’s not much faster than the original, ~20% faster according to %timeit in IPython. On the other hand, there’s a lot of (probably unnecessary, ultimately) transposing and reshaping going on.

For the record, I wrapped it up in a dummy class and filled some 3 x 3 x 3 arrays with random numbers to test:

import numpy as np

class TestClass(object):
    def interpolate_to_distance(self, distance):
        dshape = self.dist.shape
        dist = self.dist.T.reshape(-1, dshape[-1])
        data = self.data.T.reshape(-1, dshape[-1])
        intdata = np.array([np.interp(distance, di, da)
                            for di, da in zip(dist, data)])
        return intdata.reshape(dshape[0:2]).T

    def interpolate_to_distance_old(self, distance):
        interpolated_data=np.ndarray(self.dist.shape[1:])
        for j in range(interpolated_data.shape[1]):
            for i in range(interpolated_data.shape[0]):
                interpolated_data[i,j]=np.interp(
                           distance,self.dist[:,i,j],self.data[:,i,j])
        return(interpolated_data)

if __name__ == '__main__':
    testobj = TestClass()

    testobj.dist = np.random.randn(3, 3, 3)
    testobj.data = np.random.randn(3, 3, 3)

    distance = 0
    print 'Old:\n', testobj.interpolate_to_distance_old(distance)
    print 'New:\n', testobj.interpolate_to_distance(distance)

Which prints (for my particular set of randoms):

Old:
[[-0.59557042 -0.42706077  0.94629049]
 [ 0.55509032 -0.67808257 -0.74214045]
 [ 1.03779189 -1.17605275  0.00317679]]
New:
[[-0.59557042 -0.42706077  0.94629049]
 [ 0.55509032 -0.67808257 -0.74214045]
 [ 1.03779189 -1.17605275  0.00317679]]

I also tried np.vectorize(np.interp) but couldn’t get that to work. I suspect that would be much faster if it did work.

I couldn’t get np.fromfunction to work either, as it passed (2) 3 x 3 (in this case) arrays of indices to np.interp, the same arrays you get from np.mgrid.

One other note: according the the docs for np.interp,

np.interp does not check that the x-coordinate sequence xp is increasing. If
xp is not increasing, the results are nonsense. A simple check for
increasingness is::

np.all(np.diff(xp) > 0)

Obviously, my random numbers violate the ‘always increasing’ rule, but you’ll have to be more careful.