Question

I have a 200,000 line file that I’m trying to import, but I get an error:

Error in scan(file, what, nmax, sep, dec, quote, skip, nlines, na.strings,  : 
  line 39194 did not have 10 elements

Looking at that line with less -N (just type “39000e” to skip to the line you want) I can’t see a difference, so I used split -l 30000 optimized_bail_1127.csv to break it into chunks so I can try importing smaller pieces and appending them (or examining them).

split produces a bunch of files named “xaa”, “xab”, … xag etc. So I want to do something like:

files <- dir(pattern="xa[a-g]")
for(f in files) {
  print (f)
  f <- read.table(f,sep = '|')
}

to get tables for each subfile. I’d also like it to just skip errors (print them, but keep going) so I can at least see how many of these I can import cleanly.

Answer 1

In order to see error messages but keep going in a loop, you can do try():

FSU <- function() {
       for(i in 1:100){
            if(i == 23) lm(NA~NA)
            }
       print("you made it!")
       }

> FSU()
Error in terms.formula(formula, data = data) : 
  invalid term in model formula

Using try()

DontFSU <- function() {
       for(i in 1:100){
            if(i == 23) try(lm(NA~NA))
            }
       print("you made it!")
       }

> DontFSU()
Error in terms.formula(formula, data = data) : 
   invalid term in model formula
[1] "you made it!"