Question

I’m using psycopg2 to pull data from a DB to get a date, and time where a failure was logged. I would like to graph it where the time is on the y axis, and the date is on the x axis. A dot graph seems to be ideal for this.

This is part of the code where I get the data, and use zip on it to make the two objects I want to graph,

    cur.execute("SELECT date, time FROM querytimes where server = (%s) AND date > CURRENT_DATE - (%s) AND fail = 'Yes'", (server,days,))

    # retrieve the whole result set
    data = cur.fetchall()
    # close connection
    cur.close()
    conn.close()
    # Test
    date, time = zip(*data)

These are the resulting "date" and time" objects,

(datetime.date(2013, 1, 10), datetime.date(2013, 1, 10), datetime.date(2013, 1, 10), datetime.date(2013, 1, 10), datetime.date(2013, 1, 10), datetime.date(2013, 1, 10), datetime.date(2013, 1, 10), datetime.date(2013, 1, 10), datetime.date(2013, 1, 10), datetime.date(2013, 1, 10), datetime.date(2013, 1, 10), datetime.date(2013, 1, 10), datetime.date(2013, 1, 10), datetime.date(2013, 1, 10), datetime.date(2013, 1, 10), datetime.date(2013, 1, 10), datetime.date(2013, 1, 11), datetime.date(2013, 1, 12), datetime.date(2013, 1, 12), datetime.date(2013, 1, 13))

(datetime.time(15, 25, 3), datetime.time(15, 26, 3), datetime.time(15, 27, 4), datetime.time(15, 28, 3), datetime.time(15, 29, 3), datetime.time(15, 30, 4), datetime.time(15, 31, 3), datetime.time(15, 32, 4), datetime.time(15, 33, 3), datetime.time(15, 34, 3), datetime.time(15, 35, 3), datetime.time(15, 36, 3), datetime.time(15, 37, 3), datetime.time(15, 38, 3), datetime.time(15, 39, 3), datetime.time(15, 40, 3), datetime.time(0, 20, 4), datetime.time(6, 19, 3), datetime.time(10, 50, 3), datetime.time(2, 19, 3))

I am having problems graphing this. From what I understand these need converted to floats first before matplotlib to understand the data?

Tried using date2num and that works on the "date" object, but not sure how to do that for the "time" object.

Thanks

Answer 1

You could achieve this easily using Pandas.

import pandas as pd
ts = pd.Series(times, index=dates)
ts.plot(rot=30, style='x-')

to get something like

Alternative approach: datetimes as indices

Having datetime objects as your index, not just the date objects sometimes allows for more flexibility, so personally I would represent this kind of data that way. You could do it by first combining your date and time objects

datetimes = []
for date, time in zip(dates, times):
    datetimes.append(datetime.datetime.combine(date, time))

building your TimeSeries

ts = pd.Series(1, index=datetimes)

(1 is to indicate one failure at that moment)

Now you could create another useful view on this kind of data which enables you to see the quantity per day.

ts.groupby(lambda x : x.date()).sum().plot(rot=30, style='x-', kind='bar')