Question

I have object d connected to h5 dataset:

>>> data = d[:, :, 0].astype(np.float32)
>>> data.shape
(17201, 10801)
>>> data[data==-32768] = data[data>0].min()
Traceback (most recent call last):
  File "<interactive input>", line 1, in <module>
MemoryError

Can I do some other slicing trick to avoid this error?

Answer 1

OK, I’m writing answer myself, as there is acceptable solution, gained after @mgilson questioned data type.

If data allows, memory error can be avoided by using simpler data type while operating on array. Considering initial question this worked for me:

>>> data = d[:, :, 0].astype(np.short)
>>> data[data==-32768] = data[data>0].min()
>>> data = data.astype(np.float32)