Question

I’m currently exploring the python library pandas with some hands-on data, where one of the columns contains a datetime object. However, when a table is parsed using the DataFrame method, the datetime objects in the date column are parsed to an initial value e.g. 1970-01-16 14:12:28.

If I for instance have a np.array with following content:

np.array(result, dtype=my_dtype) =
array([ (datetime.datetime(2012, 9, 9, 0, 0), datetime.datetime(2012, 9, 8, 15, 10)),
dtype=[('Date', ('<M8[us]', {})), ('Forecasting', ('<M8[us]', {})),

when parsed, it will return following:

test = pandas.DataFrame(np.array(result, dtype=my_dtype))
test['Date'] =
1970-01-16 14:12:28.800000

and

test['Forecasting'] =
1970-01-16 14:11:57

Is this a bug, or what am I doing wrong?

FYI: pandas.version = 0.8.1, numpy.version = 1.6.2 and Python 2.7.3

Answer 1

I’m not familiar with panda, but I suspect something is wrong with the dtype you’re choosing for your fields. It looks like you’re using [us], that is, microseconds from the epoch (1970-01-01), when you probably should be using days from the epoch.

Another possibility is to use basic datetime objects in your ndarray, by using dtype=object.