I have an irregular time-series (with DateTime and RainfallValue) in a csv file C:\SampleData.csv:
DateTime,RainInches
1/6/2000 11:59,0
1/6/2000 23:59,0.01
1/7/2000 11:59,0
1/13/2000 23:59,0
1/14/2000 0:00,0
1/14/2000 23:59,0
4/14/2000 3:07,0.01
4/14/2000 3:12,0.03
4/14/2000 3:19,0.01
12/31/2001 22:44,0
12/31/2001 22:59,0.07
12/31/2001 23:14,0
12/31/2001 23:29,0
12/31/2001 23:44,0.01
12/31/2001 23:59,0.01
Note: The irregular time-steps could be 1 min, 15 min, 1 hour, etc. Also, there could be multiple observations in a desired 15-min interval.
I am trying to create a regular 15-minute time-series from 2000-01-01 to 2001-12-31 that should look like:
2000-01-01 00:15:00 0.00
2000-01-01 00:30:00 0.00
2000-01-01 00:45:00 0.00
...
2001-12-31 23:30:00 0.01
2001-12-31 23:45:00 0.01
Note: The time-series is regular with 15-minute intervals, filling the missing data with 0. If there are more than one data point in a 15 minute intervals, they are summed.
Here’s is my code:
library(zoo)
library(xts)
filename = "C:\\SampleData.csv"
ReadData <- read.zoo(filename, format = "%m/%d/%Y %H:%M", sep=",", tz="UTC", header=TRUE) # read .csv as a ZOO object
RawData <- aggregate(ReadData, index(ReadData), sum) # Merge duplicate time stamps and SUM the corresponding data (CAUTION)
RawDataSeries <- as.xts(RawData,order.by =index(RawData)) #convert to an XTS object
RegularTimes <- seq(as.POSIXct("2000-01-01 00:00:00", tz = "UTC"), as.POSIXct("2001-12-31 23:45:00", tz = "UTC"), by = 60*15)
BlankTimeSeries <- xts((rep(0,length(RegularTimes))),order.by = RegularTimes)
MergedTimeSeries <- merge(RawDataSeries,BlankTimeSeries)
TS_sum15min <- period.apply(MergedTimeSeries,endpoints(MergedTimeSeries, "minutes", 15), sum, na.rm = TRUE )
TS_align15min <- align.time( TS_sum15min [endpoints(TS_sum15min , "minutes", 15)], n=60*15)
Problem: The output time series TS_align15min:
(a) has repeating blocks of time-stamps
(b) starts (mysteriously) from 1999, as:
1999-12-31 19:15:00 0 1999-12-31 19:30:00 0 1999-12-31 19:45:00 0 1999-12-31 20:00:00 0 1999-12-31 20:15:00 0 1999-12-31 20:30:00 0
What am I doing wrong?
Thank you for any direction!
xts extends zoo, and zoo has extensive examples for this in its vignettes and documentation.
Here is a worked example. I think I have done that more elegantly in the past, but this is all I am coming up with now:
A two hour time grid, and some random observations leaving some cells empty and some
filled.
That is a 15 minutes grid aggregation but not on our time grid.
New xts object, and merged object. Now use
na.locf()to carry the observationsforward:
And then we can merge again as an inner join on the time-grid xts
twoh: