Question

Suppose my program creates a large array of data which I then save with numpy’s savez routine. However, I’d also like to store some additional information together with that array. Examples would be the git commit id of the current version, and the input parameters used to generate the data so that later I can look at the data and know exactly how I created it.

Is there a way to save this information directly together with the array in a npz file, or would I have to create a separate file?

Answer 1

You should be able to:

In [2]: a = np.arange(10)

In [3]: b = 'git push'

In [5]: np.savez('file',a=a,b=b)

In [7]: data = np.load('file.npz')

In [8]: data.keys()
Out[8]: ['a', 'b']

In [9]: data['a']
Out[9]: array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])

In [10]: str(data['b'])
Out[10]: 'git push'

So you can save arbitrary named data and get a dictionary-like object out. Perhaps a better format to use that may be more flexible and has built in support for all sorts of metadata is hdf5 using either h5py or pytables:

http://h5py.alfven.org/docs/

http://www.pytables.org/