Question

Let’s say you have a data frame generated by the following commands:

date <- seq(as.Date("2012-09-01"), Sys.Date(), 1)
id <- rep(c("a","b","c","d"), 8)
bdate <- seq(as.Date("2012-08-01"), as.Date("2012-11-01"), 1)[sample(1:32, 32)]

# The end date should be random but greater than the begin date. However, I set it to 15 days later for simplicity.
edate <- bdate + 15

value <- seq(1, 1000, 1)[sample(1:1000, 32)]
dfa <- data.frame(id, value, bdate, edate)
names(dfa) <- c("ID", "Value", "Begin.Date", "End.Date")

The goal is to sum all the observations by ID (i.e., “a”, “b”, or “c”) in the following way:

Date        a   b   c
2012-08-01  XXX YYY ZZZ
2012-08-02  XXX YYY ZZZ
2012-08-03  XXX YYY ZZZ

The values XXX, YYY, and ZZZ represent the sum of all the observations where the date on column “Date” falls between dfa$Begin.Date and dfa$End.Date on the original data frame, for each ID.

My current solution is practically useless for large datasets so I was wondering if there are any faster ways to do it.

My current script:

# Create additional data frame
dfb <- data.frame(seq(as.Date("2012-08-01"), as.Date("2012-11-01"), 1))
names(dfb)[1] <- "Date"

# Variable for unique IDs
nid <- unique(dfa$ID)

# Number of total IDs
tid <- length(nid)

for (i in c(1:tid))
{
sums <- vapply(dfb$Date, function(x)
{
temp <- subset(dfa, dfa$ID == nid[i])
temp <- subset(temp, temp$Begin.Date < x & temp$End.Date > x)
res <- sum(temp$Value)
res
}, FUN.VALUE = 0.1
)
dfb[1+i] <- sums
}

# Change column names to ID
names(dfb) <- c("Date", as.character(nid))

EDIT: I posted an answer below with a more efficient way to do this. However, I accepted Matthew’s answer because it set me on the right path.

Answer 1

Interesting. It seems very similar to this question :

Splitting irregular time series into regular monthly averages

Does that help? There, as in your question, one trick can be to join to the prevailing begin using roll=TRUE in package data.table. Especially since you stated you have large datasets.