Question

HDF datasets from h5py implement a subset of the functionality of numpy arrays, but have the advantage that only the data you actually access will be read into memory. I therefore want to work with datasets for as long as I can, and only convert them into arrays when I need some functionality that they lack. To that end, I’ve tried to define a wrapper class which initially contains a dataset and forwards everything to that, but which catches name errors and converts its dataset into an array when this happens. My current implementation is:

class DArr:
    def __init__(self, dset):
        self.arr = dset
    def __getitem__(self, args):
        try:
            return self.arr.__getitem__(args)
        except:
            self.arr = np.array(self.arr)
            return self.arr.__getitem__(args)
    def __getattr__(self, name):
        try:
            return self.arr.__getattr__(name)
        except:
            self.arr = np.array(self.arr)
            return self.arr.__getattr__(name)

However, this fails when self.arr has become a numpy.array, as these apparently do not have a __getattr__ I can forward to. What is the correct way to do this kind of forwarding? The goal is that a DArr should behave just like a numpy.array from the user’s point of view.

Answer 1

Use the getattr builtin function:

def __getattr__(self, name):
    try:
        return getattr(self.arr, name)
    except:
        self.arr = np.array(self.arr)
        return getattr(self.arr, name)

For __getitem__ use the [] indexing operator:

def __getitem__(self, args):
    try:
        return self.arr[args]
    except:
        self.arr = np.array(self.arr)
        return self.arr[args]