Question

I have been trying to import several csv files, use the function “melt” and merge them into a single database in R. All the files have an “id”, “date.time” and “tag” column; however, the rest of the columns differ among files. This is an example of a few lines in one the file:

               date.time   tag 111015 111016 113949 113950
1 1 2012-10-11 00:00:00 14767      0      0      0      0
2 2 2012-10-11 01:00:00 14767      0      0      0      0
3 3 2012-10-11 02:00:00 14767      0      0      0      0
4 4 2012-10-11 03:00:00 14767      0      0      0      0
5 5 2012-10-11 04:00:00 14767      0      0      0      0
6 6 2012-10-11 05:00:00 14767      0      0      0      0

library(reshape2)

# Import files

files<-list.files()
data<-lapply(files,read.csv,header=TRUE,sep=",",check.names=FALSE)

I am trying to use this loop to melt each file and bind the resulting data frame. However, its only working for the last file in the loop. I don’t know exactly how to set the loop/function so that it can perform first the “melt” of each file and them “merge/bind” them into a single data frame.

for(j in 1:length(data)){
   dm<-melt(data[[j]],measure.vars=c(4:length(data[[j]])),
     id=c("date.time","tag"),variable.name="receiver")

   results<-rbind(dm)   

  }

Any suggestions will be appreciated!

Answer 1

Its better to do using lapply to load everything first and then use melt as follows: (Assuming all your files are in the variable files,

Note: Untested 
require(reshape2)
files <- list.files(my.dir, full.names = TRUE)
# first load all files
dd <- lapply(1:length(files), function(idx) {
    d <-read.csv(files[idx], header = TRUE, sep=",", check.names = FALSE)
    # if you want the file index
    d$file.idx <- idx
    d
})
# merge all
dd <- do.call(rbind, dd)
# now melt
dd.m <- melt(dd, c(4:length(d)), c("date.time","tag"), variable.name = "receiver")

Edit: After Op’s edit

Note: Untested 
require(reshape2)
files <- list.files(my.dir, full.names = TRUE)
dd.m <- lapply(1:length(files), function(idx) {
    # load the file
    d <-read.csv(files[idx], header = TRUE, sep=",", check.names = FALSE)
    # now melt immediately
    d.m <- melt(d, c("date.time","tag"), c(4:length(d)))
})
# merge all
dd.m <- do.call(rbind, dd.m)