Question

Possible Duplicate:
Beginner tips on using plyr to calculate year-over-year change across groups

What is a good way to calcualte a year-on-year difference (new variable) for an existing data frame variable (i.e. sales) across multiple variable groups (i.e. Region and Food)?

Below is a example of the data frame structure:

Date              Region    Type    Sales

1/1/2001    East    Food    120
1/1/2001    West    Housing 130
1/1/2001    North   Food    130
1/2/2001    East    Food    133
1/3/2001    West    Housing 140
1/4/2001    North   Food    150
….
….
1/29/2013   East    Food    125
1/29/2013   West    Housing 137
1/29/2013   North   Food    1350

Also, in addition to differening the data, I would like to calcuate a a trailing (say 7 day) moving average.

Any guidance would be greatly appreciated.

Answer 1

Here is something to get you started. data.table is a great package for this sort of things as it provides a concise and easy-to-use syntax (once you are past the learning curve) for these kinds of things.

library(data.table)

Create a reproducible example

set.seed(128)
regions = c("East", "West", "North", "South")
types = c("Food", "Housing")
dates <- seq(as.Date('2009-01-01'), as.Date('2011-12-31'), by = 1)
n <- length(dates)
dt <- data.table(Date = dates, 
                 Region = sample(regions, n, replace = TRUE),
                 Type = sample(types, n, replace = TRUE),
                 Sales = round(rnorm(n, mean = 100, sd = 10)))

Add Year column

dt[, Year := year(Date)]

> dt
        Date Region    Type Sales Year
1: 2009-01-01   West    Food   119 2009
2: 2009-01-02  North Housing   102 2009
3: 2009-01-03  North Housing   102 2009
4: 2009-01-04  North    Food   101 2009
5: 2009-01-05   West    Food   101 2009
---                                     
1091: 2011-12-27   East Housing   122 2011
1092: 2011-12-28   East Housing    88 2011
1093: 2011-12-29  North    Food   115 2011
1094: 2011-12-30   West Housing    96 2011
1095: 2011-12-31   East    Food   101 2011

Calculate summary by year

summary <- dt[, list(Sales = sum(Sales)), by = 'Year,Region,Type']
setkey(summary, 'Year')

> head(summary)
Year Region    Type Sales
1: 2009   West    Food  4791
2: 2009  North Housing  3517
3: 2009  North    Food  6774
4: 2009  South Housing  4380
5: 2009   East    Food  4144
6: 2009   West Housing  4275

Function to create year-on-year diffs for each region/product combo.

YoYdiff <- function(dt) {
  # Calculate year-on-year difference for Sales column
  data.table(Sales.Diff = diff(dt$Sales), Year = dt$Year[-1])
}

Calculate year-on-year difference by column. This works for my example as setkey(dt, Year) sorts the data table by Year, but if your example misses some years for some products/regions you have to be more careful.

> summary[, YoYdiff(.SD), by = 'Region,Type']
    Region    Type Sales.Diff Year
 1:   West    Food       -412 2010
 2:   West    Food        121 2011
 3:  North Housing       1907 2010
 4:  North Housing      -1457 2011
 5:  North    Food      -3087 2010
 6:  North    Food        369 2011
 7:  South Housing       -539 2010
 8:  South Housing        575 2011
 9:   East    Food       1264 2010
10:   East    Food      -1732 2011
11:   West Housing        298 2010
12:   West Housing       -410 2011
13:  South    Food       -889 2010
14:  South    Food       1045 2011
15:   East Housing       1146 2010
16:   East Housing       1169 2011