Question

I have a csv file with data looking like (see below). I need help parsing datetime and fill missing date_time and assigning missing data as “M”(missing):

Datetime, Data

19920101 00:00,2
19920101 01:00,3
19920101 23:00,5
19920505 12:00,5

Answer 1

Not a complete answer but just tried to parse the date time string

>>> s="19920101 00:00"
>>> format = "%Y%m%d %H:%M"
>>> d = datetime.datetime.strptime(s, format)
>>> print d
1992-01-01 00:00:00

Will this help in finding out the missing dates and times for you.

I could not understand what 3 is in the string 3 19920101 23:00.

[Edit: based on your comment]

>>> expected = d + datetime.timedelta(days=1)
>>> print expected
1992-01-02 00:00:00

So in your code, you could try something like this (You will need to work and refine this)

[Edit: Code replaced]

import csv
import sys
import datetime
import pprint


all_data_points = {}
all_dates = []
expected = ''
format = "%Y%m%d %H:%M"

with open('datafile', 'rt') as f:
    reader = csv.reader(f)
    for row in reader:
        if row and 'Datetime' not in row:
            day_str = row[0]
            rain_str = row[1]
            if not expected:
                all_data_points[day_str] = rain_str
                all_dates.append(day_str)
                d = datetime.datetime.strptime(day_str, format)
                expected = d + datetime.timedelta(days=1)
            else:
                d = datetime.datetime.strptime(day_str, format)
                gap_in_days = d - expected 
                start_day = expected
                if gap_in_days.days > 1:
                    for i in xrange(gap_in_days.days):
                        next_day = start_day + datetime.timedelta(days=1+i)
                        day_str = next_day.strftime(format)
                        all_data_points[day_str] = 'M'
                        all_dates.append(day_str)
                all_data_points[day_str] = rain_str
                expected = d

    pprint.pprint(all_data_points)