Question

I have been using data.table for some computation and am wondering what are the possible return types for the j parameter so that it stacks up my output correctly? I know data.frame is acceptable so list must be as well? My function returns multiple rows and multiple columns for each id. So imagine:

dtb <- data.table(id=rep(1:5,20), a=1:100, b=sample(1:100, 100), c=sample(1:100, 100))
f <- function(dt) { return(c(dt$a+1, dt$b+1, dt$c+1))}
dtb[,f(.SD), by=id]

This clearly does not work properly. This does:

dtb <- data.table(id=rep(1:5,20), a=1:100, b=sample(1:100, 100), c=sample(1:100, 100))
f <- function(dt) { return(data.frame(a=dt$a+1, b=dt$b+1, c=dt$c+1))}
dtb[,f(.SD), by=id]

Constructing these data.frames seems like a really inefficient way to do things. What are some suggestions? The by must be used.

Answer 1

When you wrote this c(dt$a+1, dt$b+1, dt$c+1) you should have expected a single vector (plus the group id column. Try this instead:

dtb <- data.table(id=rep(1:5,20), a=1:100, b=sample(1:100, 100), c=sample(1:100, 100))
f <- function(dt) { return(list(dt$a+1, dt$b+1, dt$c+1))}
dtb[,f(.SD), by=id]

EDIT2 (there was an error in my earlier edit that I only noticed when posting the full code). To the question about “cheaper”: Here’s a benchmark run that shows list construction to be ‘cheaper’:

flst <- function(dt) { return(list(dt$a+1, dt$b+1, dt$c+1))}
fdf <- function(dt) { return(data.frame(dt$a+1, dt$b+1, dt$c+1))}
require(rbenchmark)
 benchmark(
    lstdt=dtb[ , flst(.SD), by=id], 
    dfdt=dtb[ , fdf(.SD), by=id], 
    replications=10^2
    )
   test replications elapsed relative user.self sys.self user.child sys.child
2  dfdt          100   0.466  2.89441     0.457    0.010          0         0
1 lstdt          100   0.161  1.00000     0.159    0.003          0         0