Home/ Questions/Q 5982177
In Process

The Archive Base Latest Questions

Editorial Team
  • 0
Asked: 2026-05-22T22:02:58+00:00

Total newbie to R here (so far very pleased!). I have a large array(vector?)

  • 0

Total newbie to R here (so far very pleased!).

I have a large array(vector?) of date information, and I would like to remove the rows that do not fit within an my date range of interest (which is 24-sept-2003 to 10-december-2003, from 12pm to 6pm each day). I seem to be running out of memory

Here’s an example of the issue as well as I can put it:

> head(p_times)
[1] "2001-04-11 07:57:27 EDT" "2001-04-11 08:18:11 EDT"
[3] "2001-04-11 08:21:33 EDT" "2001-04-11 08:22:52 EDT"
[5] "2001-04-11 08:25:39 EDT" "2001-04-11 08:31:18 EDT"
> length(p_times)
[1] 31164014
> class(p_times)
[1] "POSIXlt" "POSIXt" 
> fp_times = p_times[p_times$year==103]
R(59593,0xa0506540) malloc: *** mmap(size=249315328) failed (error code=12)
*** error: can't allocate region
*** set a breakpoint in malloc_error_break to debug
R(59593,0xa0506540) malloc: *** mmap(size=124657664) failed (error code=12)
*** error: can't allocate region
*** set a breakpoint in malloc_error_break to debug
R(59593,0xa0506540) malloc: *** mmap(size=57901056) failed (error code=12)
*** error: can't allocate region
*** set a breakpoint in malloc_error_break to debug

I was planning to filter by year to remove most of the information, and then using the same approach to filter by month / day / hour

Share

Leave an answer

You must login to add an answer.

Need An Account,

1 Answer

  1. Editorial Team

    Could you use POSIXct instead of POSIXlt? POSIXlt objects seem to take ~5x more memory than POSIXct objects:

    > set.seed(21)
> x <- Sys.time() - trunc(runif(31164014)*3e8); range(x)
[1] "2001-12-03 11:55:25 CST" "2011-06-06 18:15:07 CDT"
> print(object.size(x), units="Mb")
237.8 Mb
> print(object.size(as.POSIXlt(x)), units="Mb")
1188.8 Mb
> 1188.8/237.8
[1] 4.999159
> sessionInfo()
R version 2.13.0 (2011-04-13)
Platform: x86_64-pc-linux-gnu (64-bit)

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C              
 [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8    
 [5] LC_MONETARY=C              LC_MESSAGES=en_US.UTF-8   
 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                 
 [9] LC_ADDRESS=C               LC_TELEPHONE=C            
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base
    • 0