Question

Suppose there’s a sensor which records the date and time at every activation. I have this data stored as a list in a .json file in the format (e.g.) “2000-01-01T00:30:15+00:00”.

Now, what I want to do is import this file in python and use NumPy/ Mathplotlib to plot how many times this sensor is activated per day.

My problem is, using this data, I don’t know how to write an algorithm which counts how many times the sensor is activated daily. (This should be simple, but due to limited Python knowledge, I’m stuck). Supposedly there is a way to split this list wrt T, bin each recording by date (e.g. “2000-01-01”) and then count the recordings on this date.

How would you count how many times the sensor is activated? (to then make a plot showing the number of activations each day?)

Answer 1

First of all you need to load your JSON file:

import json
with open("logfile.json", "r") as logfile:
    records = json.load(logfile)

Records will be a list or dictionary containing your records.

Assuming that your logfile looks like:

[u"2000-01-01T00:30:15+00:00",
 u"2000-01-01T00:30:16+00:00",
 ...
 ]

Records will be a list of strings. So parsing the dates is just:

import datetime
for record in records:
    datepart, _ = record.split("T")
    date = datetime.datetime.strptime(datepart, "%Y-%m-%d")

Hopefully that’s clear enough. Using “string”.split and datetime.strptime should do the trick, although you don’t have to parse this into a date object just to bin it but it may make things easier later on.

Finally, binning should be pretty straightforward using a dictionary of lists. Starting
from what we’ve got above let’s add binning:

import collections
import datetime
date_bins = collections.defaultdict(list)
for record in records:
    datepart, _ = record.split("T")
    date = datetime.datetime.strptime(datepart, "%Y-%m-%d")
    date_bins[date].append(record)

This should give you a dictionary where each key is a date and each value is the list of records that were logged on that day.

You’ll probably want to sort this by date (although you may be able to use collections.OrderedDict if the data is already in order).

Counting activations per day could be something like:

for date in date_bins:
    print "activations on %s: %s"%(date, len(date_bins[date]))

Of course it’s a little bit more work to take that information and massage it into a format that matplotlib needs but it shouldn’t be too bad from here.