Question

I am tryng to understand how to set dtypes of an array. My original numpy array dimensions are (583760, 7) i.e. 583760 rows and 7 columns. I am setting dtype as follows

>>> allRics.shape
(583760, 7)
>>> allRics.dtype = [('idx', np.float), ('opened', np.float), ('time', np.float),('trdp1',np.float),('trdp0',np.float),('dt',np.float),('value',np.float)]
>>> allRics.shape
(583760, 1)

Why is there a change in the original shape of the array? What causes this change? I am basically trying to sort original numpy array by time column and thats why I am setting the dtype. But after the dimension change, I am not able to sort array

>>> x=np.sort(allRics,order='time')

there is no change in the output of the above command. Could you please advice?

Answer 1

You are turning your array into a structured array. Basically, instead of a 2D array it is now treated as a 1D array of structs. Take a look as a simpler example below:

>>> import numpy as np
>>> arr = np.array([(1,2,3),(3,4,5)])
>>> arr
array([[1, 2, 3],
       [3, 4, 5]])
>>> arr.shape
(2, 3)
>>> arr.dtype=[('a',int),('b',int),('c', int)]
>>> arr  # Notice that tuples inside the elements
array([[(1, 2, 3)],
       [(3, 4, 5)]], 
      dtype=[('a', '<i8'), ('b', '<i8'), ('c', '<i8')])
>>> arr.shape
(2, 1)

The structured array not sorting is most assurdly a bug. It looks like a work around it so actually declare the array a structured array to begin with:

>>> arr_s = np.sort(arr, order='b')
>>> arr_s
array([[(1, 2, 3)],
       [(3, 4, 5)]], 
      dtype=[('a', '<i8'), ('b', '<i8'), ('c', '<i8')])
>>> dtype=[('a',np.int64),('b',np.int64),('c', np.int64)]
>>> arr = np.array([(5,2,3),(3,4,1)], dtype=dtype)
>>> arr
array([(5, 2, 3), (3, 4, 1)], 
      dtype=[('a', '<i8'), ('b', '<i8'), ('c', '<i8')])
>>> arr_s = np.sort(arr, order='a')
>>> arr_s
array([(3, 4, 1), (5, 2, 3)], 
      dtype=[('a', '<i8'), ('b', '<i8'), ('c', '<i8')])
>>> arr_s = np.sort(arr, order='b')
>>> arr_s
array([(5, 2, 3), (3, 4, 1)], 
      dtype=[('a', '<i8'), ('b', '<i8'), ('c', '<i8')])
>>> arr_s = np.sort(arr, order='c')
>>> arr_s
array([(3, 4, 1), (5, 2, 3)], 
      dtype=[('a', '<i8'), ('b', '<i8'), ('c', '<i8')])
>>>