Home/ Questions/Q 6710597
In Process

The Archive Base Latest Questions

Editorial Team
  • 0
Asked: 2026-05-26T08:02:27+00:00

I’ve begun to believe that data frames hold no advantages over matrices, except for

  • 0

I’ve begun to believe that data frames hold no advantages over matrices, except for notational convenience. However, I noticed this oddity when running unique on matrices and data frames: it seems to run faster on a data frame.

a   = matrix(sample(2,10^6,replace = TRUE), ncol = 10)
b   = as.data.frame(a)

system.time({
    u1 = unique(a)
})
 user  system elapsed
1.840   0.000   1.846


system.time({
    u2 = unique(b)
})
 user  system elapsed
0.380   0.000   0.379

The timing results diverge even more substantially as the number of rows is increased. So, there are two parts to this question.

  1. Why is this slower for a matrix? It seems faster to convert to a data frame, run unique, and then convert back.

  2. Is there any reason not to just wrap unique in myUnique, which does the conversions in part #1?

Note 1. Given that a matrix is atomic, it seems that unique should be faster for a matrix, rather than slower. Being able to iterate over fixed-size, contiguous blocks of memory should generally be faster than running over separate blocks of linked lists (I assume that’s how data frames are implemented…).

Note 2. As demonstrated by the performance of data.table, running unique on a data frame or a matrix is a comparatively bad idea – see the answer by Matthew Dowle and the comments for relative timings. I’ve migrated a lot of objects to data tables, and this performance is another reason to do so. So although users should be well served to adopt data tables, for pedagogical / community reasons I’ll leave the question open for now regarding the why does this take longer on the matrix objects. The answers below address where does the time go, and how else can we get better performance (i.e. data tables). The answer to why is close at hand – the code can be found via unique.data.frame and unique.matrix. 🙂 An English explanation of what it’s doing & why is all that is lacking.

Share

Leave an answer

You must login to add an answer.

Need An Account,

1 Answer

  1. Editorial Team

    1. In this implementation, unique.matrix is the same as unique.array

      > identical(unique.array, unique.matrix)

      [1] TRUE

    2. unique.array has to handle multi-dimensional arrays which requires additional processing to ‘collapse’ the extra dimensions (those extra calls to paste()) which are not needed in the 2-dimensional case. The key section of code is:

      collapse <- (ndim > 1L) && (prod(dx[-MARGIN]) > 1L)

      temp <- if (collapse)
      apply(x, MARGIN, function(x) paste(x, collapse = "\r"))

    3. unique.data.frame is optimised for the 2D case, unique.matrix is not. It could be, as you suggest, it just isn’t in the current implementation.

    Note that in all cases (unique.{array,matrix,data.table}) where there is more than one dimension it is the string representation that is compared for uniqueness. For floating point numbers this means 15 decimal digits so

    NROW(unique(a <- matrix(rep(c(1, 1+4e-15), 2), nrow = 2)))

    is 1 while

    NROW(unique(a <- matrix(rep(c(1, 1+5e-15), 2), nrow = 2)))

    and

    NROW(unique(a <- matrix(rep(c(1, 1+4e-15), 1), nrow = 2)))

    are both 2. Are you sure unique is what you want?

    • 0