Question

What I need:

I have a huge data frame with the following columns (and some more, but these are not important). Here’s an example:

    user_id video_id group_id    x   y
1         1        0        0   39 108
2         1        0        0   39 108
3         1       10        0  135 180
4         2        0        0   20 123

User, video and group IDs are factors, of course. For example, there are 20 videos, but each of them has several “observations” for each user and group.

I’d like to transform this data frame into the following format, where there are as many x.N, y.N as there are users (N).

video_id  x.1   y.1  x.2  y.2  …
       0   39   108   20  123

So, for video 0, the x and y values from user 1 are in columns x.1 and y.1, respectively. For user 2, their values are in columns x.2, y.2, and so on.

What I’ve tried:

I made myself a list of data frames that are solely composed of all the x, y observations for each video_id:

summaryList = dlply(allData, .(user_id), function(x) unique(x[c("video_id","x","y")]) )

That’s how it looks like:

List of 15
 $ 1 :'data.frame': 20 obs. of  3 variables:
  ..$ video_id: Factor w/ 20 levels "0","1","2","3",..: 1 11 8 5 12 9 20 13 7 10 ...
  ..$ x       : int [1:20] 39 135 86 122 28 167 203 433 549 490 ...
  ..$ y       : int [1:20] 108 180 164 103 187 128 185 355 360 368 ...
 $ 2 :'data.frame': 20 obs. of  3 variables:
  ..$ video_id: Factor w/ 20 levels "0","1","2","3",..: 2 14 15 4 20 6 19 3 13 18 ...
  ..$ x       : int [1:20] 128 688 435 218 528 362 299 134 83 417 ...
  ..$ y       : int [1:20] 165 117 135 179 96 328 332 563 623 476 ...

Where I’m stuck:

What’s left to do is:

• Merge each data frame from the summaryList with each other, based on the video_id. I can’t find a nice way to access the actual data frames in the list, which are summaryList[1]$`1`, summaryList[2]$`2`, et cetera.

  @James found out a partial solution:

  Reduce(function(x,y) merge(x,y,by="video_id"),summaryList)
  

• Ensure the column names are renamed after the user ID and not kept as-is. Right now my summaryList doesn’t contain any info about the user ID, and the output of Reduce has duplicate column names like x.x y.x x.y y.y x.x y.x and so on.

How do I go about doing this? Or is there any easier way to get to the result than what I’m currently doing?

Answer 1

Reduce does the trick:

reducedData <- Reduce(function(x,y) merge(x,y,by="video_id"),summaryList)

… but you need to fix the names afterwards:

names(reducedData)[-1] <- do.call(function(...) paste(...,sep="."),expand.grid(letters[24:25],names(summaryList)))

The result is:

   video_id  x.1 y.1  x.2 y.2  x.3 y.3  x.4 y.4  x.5 y.5  x.6 y.6  x.7 y.7  x.8
1         0   39 108  899 132   61 357  149 298 1105 415  148 208  442 200  210
2         1 1125  70  128 165 1151 390  171 587  623 623   80 643  866 310  994