Question

I have a paneldata which looks like:

(Only the substantially cutting for my question)

Persno 122 122 122 333 333 333 333 333 444 444 
Income 1500 1500 2000 2000 2100 2500 2500 1500 2000 2200
year 1990 1991 1992 1990 1991 1992 1993 1994 1992 1993

Now I would like to give out for every row (PErsno) the years of workexperience at the begining of the year. I use ddply

hilf3<-ddply(data, .(Persn0), summarize, Bgwork = 1:(max(year) - min(year)))

To produce output looking like this:

Workexperience: 1 2 3 1 2 3 4 5 1 2

Now I want to merge the ddply results to my original panel data:

data<-(merge(data,hilf3,by.x="Persno",by.y= "Persno"))

The panel data set is very large. The code stops because of a memory size error.

Errormessage:

1: In make.unique(as.character(rows)) :

Reached total allocation of 4000Mb: see help(memory.size)

What should I do?

Answer 1

Re-reading your question, I think you don’t actually want to use merge here at all. Just sort your original data frame and rbind Bgwork from hilf3. And also, your ddply-call could perhaps result in a 1:0 sequence, which is most likely not what you want. Try

data = data[order(data$Persno, data$year),]
hilf3 = ddply(data, .(Persno), summarize, Bgwork=(year - min(year) + 1))
stopifnot(nrow(data) == nrow(hilf3))
stopifnot(all(data$Persno == hilf3$Persno))
data$Bgwork = hilf3$Bgwork